Voice and data exchange over a packet based network with echo cancellation

ABSTRACT

A signal processing system which discriminates between voice signals and data signals modulated by a voiceband carrier. The signal processing system includes a voice exchange, a data exchange and a call discriminator. The voice exchange is capable of exchanging voice signals between a switched circuit network and a packet based network. The signal processing system also includes a data exchange capable of exchanging data signals modulated by a voiceband carrier on the switched circuit network with unmodulated data signal packets on the packet based network. The data exchange is performed by demodulating data signals from the switched circuit network for transmission on the packet based network, and modulating data signal packets from the packet based network for transmission on the switched circuit network. The call discriminator is used to selectively enable the voice exchange and data exchange.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of co-pending patentApplication Atty Doc. No. 36713, filed Mar. 9, 2000, which is acontinuation-in-part of co-pending patent application Ser. No.09/493,458, filed Jan. 28, 2000, which is a continuation-in-part ofco-pending application Ser. No. 09/454,219, filed Dec. 9, 1999, priorityof each application which is hereby claimed under 35 U.S.C. §120. Thepresent application also claims priority under 35 U.S.C. §119(e) toprovisional Application No. 60/154,903, filed Sep. 20, 1999; ApplicationNo. 60/156,266, filed Sep. 27, 1999; Application No. 60/157,470, filedOct. 1, 1999; Application No. 60/160,124, filed Oct. 18, 1999;Application No. 60/161,152, filed Oct. 22, 1999; Application No.60/162,315, filed Oct. 28, 1999; Application No. 60/163,169, filed Nov.2, 1999; Application No. 60/163,170, filed Nov. 2, 1999; Application No.60/163,600; filed Nov. 4, 1999; Application No. 60/164,379, filed Nov.9, 1999; Application No. 60/164,690, filed Nov. 10, 1999; ApplicationNo. 60/164,689, filed Nov. 10, 1999; Application No. 60/166,289, filedNov. 18, 1999; Application No. 60/171,203, filed Dec. 15, 1999;Application No. 60/171,180, filed Dec. 16, 1999; Application No.60/171,169, filed Dec. 16, 1999; Application No. 60/171,184, filed Dec.16, 1999, and Application No. 60/178,258, filed Jan. 25, 2000. All theseapplications are expressly incorporated herein by reference as thoughfully set forth in full.

FIELD OF THE INVENTION

The present invention relates generally to telecommunications systems,and more particularly, to a system for interfacing telephony deviceswith packet based networks.

BACKGROUND

Telephony devices, such as telephones, analog fax machines, and datamodems, have traditionally utilized circuit switched networks tocommunicate. With the current state of technology, it is desirable fortelephony devices to communicate over the Internet, or other packetbased networks. Heretofore, an integrated system for interfacing varioustelephony devices over packet based networks has been difficult due tothe different modulation schemes of the telephony devices. Accordingly,it would be advantageous to have an efficient and robust integratedsystem for the exchange of voice, fax data and modem data betweentelephony devices and packet based networks.

SUMMARY OF THE INVENTION

In one aspect of the present invention, a method of conditioning acomposite signal, the composite signal being formed by introducing afirst signal into a second signal, includes estimating a characteristicof at least one of said first and second signals, and selectivelyconditioning the composite signal, the selection as to whether tocondition the composite signal being based on the estimatedcharacteristic.

In another aspect of the present invention, a method of cancelling a farend echo from a near end signal includes estimating a characteristic ofat least one of a far end signal and the near end signal, andselectively cancelling the echo from the near end signal, the selectionas to whether to cancel the echo from the near end signal being based onthe estimated characteristic.

In yet another aspect of the present invention, a signal conditioner forconditioning a composite signal, the composite signal being formed byintroducing a first signal into a second signal, includes a canceller tocancel the first signal from the composite signal, and a bypass toselectively enable the canceller.

It is understood that other embodiments of the present invention willbecome readily apparent to those skilled in the art from the followingdetailed description, wherein it is shown and described only embodimentsof the invention by way of illustration of the best modes contemplatedfor carrying out the invention. As will be realized, the invention iscapable of other and different embodiments and its several details arecapable of modification in various other respects, all without departingfrom the spirit and scope of the present invention. Accordingly, thedrawings and detailed description are to be regarded as illustrative innature and not as restrictive.

DESCRIPTION OF THE DRAWINGS

These and other features, aspects, and advantages of the presentinvention will become better understood with regard to the followingdescription, appended claims, and accompanying drawings where:

FIG. 1 is a block diagram of packet based infrastructure providing acommunication medium with a number of telephony devices in accordancewith a preferred embodiment of the present invention;

FIG. 2 is a block diagram of a signal processing system implemented witha programmable digital signal processor (DSP) software architecture inaccordance with a preferred embodiment of the present invention;

FIG. 3 is a block diagram of the software architecture operating on theDSP platform of FIG. 2 in accordance with a preferred embodiment of thepresent invention;

FIG. 4 is state machine diagram of the operational modes of a virtualdevice driver for packet based network applications in accordance with apreferred embodiment of the present invention;

FIG. 5 is a block diagram of several signal processing systems in thevoice mode for interfacing between a switched circuit network and apacket based network in accordance with a preferred embodiment of thepresent invention;

FIG. 6 is a system block diagram of a signal processing system operatingin a voice mode in accordance with a preferred embodiment of the presentinvention;

FIG. 7 is a block diagram of a method for canceling echo returns inaccordance with a preferred embodiment of the present invention;

FIG. 8A is a block diagram of a method for normalizing the power levelof a digital voice samples to ensure that the conversation is of anacceptable loudness in accordance with a preferred embodiment of thepresent invention;

FIG. 8B is a graphical depiction of a representative output of a peaktracker as a function of a typical input signal, demonstrating that thereference value that the peak tracker forwards to a gain calculator toadjust the power level of digital voice samples should preferably risequickly if the signal amplitude increases, but decrement slowly if thesignal amplitude decreases in accordance with a preferred embodiment ofthe present invention;

FIG. 9 is a graphical depiction of exemplary operating thresholds foradjusting the gain factor applied to digital voice samples to ensurethat the conversation is of an acceptable loudness in accordance with apreferred embodiment of the present invention;

FIG. 10 is a block diagram of a method for estimating the spectral shapeof the background noise of a voice transmission in accordance with apreferred embodiment of the present invention;

FIG. 11 is a block diagram of a method for generating comfort noise withan energy level and spectral shape that substantially matches thebackground noise of a voice transmission in accordance with a preferredembodiment of the present invention;

FIG. 12 is a block diagram of the voice decoder and the lost packetrecovery engine in accordance with a preferred embodiment of the presentinvention;

FIG. 13A is a flow chart of the preferred lost frame recovery algorithmin accordance with a preferred embodiment of the present invention;

FIG. 13B is a flow chart of the voicing decision and pitch periodcalculation in accordance with a preferred embodiment of the presentinvention;

FIG. 13C is a flow chart demonstrating voicing synthesis performed whenpackets are lost and for the first decoded voice packet after a seriesof lost packets in accordance with a preferred embodiment of the presentinvention;

FIG. 14 is a block diagram of a method for detecting dual tone multifrequency tones in accordance with a preferred embodiment of the presentinvention;

FIG. 14A is a block diagram of a method for reducing the instructionsrequired to detect a valid dual tone and for pre-detecting a dual tone;

FIG. 15 is a block diagram of a signaling service for detecting precisetones in accordance with a preferred embodiment of the presentinvention;

FIG. 16 is a block diagram of a method for detecting the frequency of aprecise tone in accordance with a preferred embodiment of the presentinvention;

FIG. 17 is state machine diagram of a power state machine which monitorsthe estimated power level within each of the precise tone frequencybands in accordance with a preferred embodiment of the presentinvention;

FIG. 18 is state machine diagram of a cadence state machine formonitoring the cadence (on/off times) of a precise tone in a voicesignal in accordance with a preferred embodiment of the presentinvention;

FIG. 18A is a block diagram of a cadence processor for detecting precisetones in accordance with a preferred embodiment of the presentinvention;

FIG. 19 is a block diagram of resource manager interface with severalVHD's and PXD's in accordance with a preferred embodiment of the presentinvention;

FIG. 20 is a block diagram of several signal processing systems in thefax relay mode for interfacing between a switched circuit network and apacket based network in accordance with a preferred embodiment of thepresent invention;

FIG. 21 is a system block diagram of a signal processing systemoperating in a real time fax relay mode in accordance with a preferredembodiment of the present invention;

FIG. 22 is a diagram of the message flow for a fax relay in non errorcontrol mode in accordance with a preferred embodiment of the presentinvention;

FIG. 23 is a flow diagram of a method for fax mode spoofing inaccordance with a preferred embodiment of the present invention;

FIG. 24 is a block diagram of several signal processing systems in themodem relay mode for interfacing between a switched circuit network anda packet based network in accordance with a preferred embodiment of thepresent invention;

FIG. 25 is a system block diagram of a signal processing systemoperating in a modem relay mode in accordance with a preferredembodiment of the present invention;

FIG. 26 is a diagram of a relay sequence for V.32bis ratesynchronization using rate re-negotiation in accordance with a preferredembodiment of the present invention;

FIG. 27 is a diagram of an alternate relay sequence for V.32bis ratesynchronization whereby rate signals are used to align the connectionrates at the two ends of the network without rate re-negotiation inaccordance with a preferred embodiment of the present invention;

FIG. 28 is a system block diagram of a QAM data pump transmitter inaccordance with a preferred embodiment of the present invention;

FIG. 29 is a system block diagram of a QAM data pump receiver inaccordance with a preferred embodiment of the present invention;

FIG. 30 is a block diagram of a method for sampling a signal of symbolsreceived in a data pump receiver in synchronism with the transmitterclock of a data pump transmitter in accordance with a preferredembodiment of the present invention;

FIG. 31 is a block diagram of a second order loop filter for reducingsymbol clock jitter in the timing recovery system of data pump receiverin accordance with a preferred embodiment of the present invention;

FIG. 32 is a block diagram of an alternate method for sampling a signalof symbols received in a data pump receiver in synchronism with thetransmitter clock of a data pump transmitter in accordance with apreferred embodiment of the present invention;

FIG. 33 is a block diagram of an alternate method for sampling a signalof symbols received in a data pump receiver in synchronism with thetransmitter clock of a data pump transmitter wherein a timing frequencyoffset compensator provides a fixed dc component to compensate for clockfrequency offset present in the received signal in accordance with apreferred embodiment of the present invention;

FIG. 34 is a block diagram of a method for estimating the timingfrequency offset required to sample a signal of symbols received in adata pump receiver in synchronism with the transmitter clock of a datapump transmitter in accordance with a preferred embodiment of thepresent invention;

FIG. 35 is a block diagram of a method for adjusting the gain of a datapump receiver (fax or modem) to compensate for variations intransmission channel conditions; and

FIG. 36 is a block diagram of a method for detecting human speech in atelephony signal.

DETAILED DESCRIPTION An Embodiment of a Signal Processing System

In a preferred embodiment of the present invention, a signal processingsystem is employed to interface telephony devices with packet basednetworks. Telephony devices include, by way of example, analog anddigital phones, ethernet phones, Internet Protocol phones, fax machines,data modems, cable modems, interactive voice response systems, PBXs, keysystems, and any other conventional telephony devices known in the art.The described preferred embodiment of the signal processing system canbe implemented with a variety of technologies including, by way ofexample, embedded communications software that enables transmission ofinformation, including voice, fax and modem data over packet basednetworks. The embedded communications software is preferably run onprogrammable digital signal processors (DSPs) and is used in gateways,cable modems, remote access servers, PBXs, and other packet basednetwork appliances.

An exemplary topology is shown in FIG. 1 with a packet based network 10providing a communication medium between various telephony devices. Eachnetwork gateway 12 a, 12 b, 12 c includes a signal processing systemwhich provides an interface between the packet based network 10 and anumber of telephony devices. In the described exemplary embodiment, eachnetwork gateway 12 a, 12 b, 12 c supports a fax machine 14 a, 14 b, 14c, a telephone 13 a, 13 b, 13 c, and a modem 15 a, 15 b, 15 c. As willbe appreciated by those skilled in the art, each network gateway 12 a,12 b, 12 c could support a variety of different telephony arrangements.By way of example, each network gateway might support any numbertelephony devices and/or circuit switched/packet based networksincluding, among others, analog telephones, ethernet phones, faxmachines, data modems, PSTN lines (Public Switching Telephone Network),ISDN lines (Integrated Services Digital Network), T1 systems, PBXs, keysystems, or any other conventional telephony device and/or circuitswitched/packet based network. In the described exemplary embodiment,two of the network gateways 12 a, 12 b provide a direct interfacebetween their respective telephony devices and the packet based network10. The other network gateway 12 c is connected to its respectivetelephony device through a PSTN 19. The network gateways 12 a, 12 b, 12c permit voice, fax and modem data to be carried over packet basednetworks such as PCs running through a USB (Universal Serial Bus) or anasynchronous serial interface, Local Area Networks (LAN) such asEthernet, Wide Area Networks (WAN) such as Internet Protocol (IP), FrameRelay (FR), Asynchronous Transfer Mode (ATM), Public Digital CellularNetwork such as TDMA (IS-13x), CDMA (IS-9x) or GSM for terrestrialwireless applications, or any other packet based system.

The exemplary signal processing system can be implemented with aprogrammable DSP software architecture as shown in FIG. 2. Thisarchitecture has a DSP 17 with memory 18 at the core, a number ofnetwork channel interfaces 19 and telephony interfaces 20, and a host 21that may reside in the DSP itself or on a separate microcontroller. Thenetwork channel interfaces 19 provide multi-channel access to the packetbased network. The telephony interfaces 23 can be connected to a circuitswitched network interface such as a PSTN system, or directly to anytelephony device. The programmable DSP is effectively hidden within theembedded communications software layer. The software layer binds allcore DSP algorithms together, interfaces the DSP hardware to the host,and provides low level services such as the allocation of resources toallow higher level software programs to run.

An exemplary multi-layer software architecture operating on a DSPplatform is shown in FIG. 3. A user application layer 26 providesoverall executive control and system management, and directly interfacesa DSP server 25 to the host 21 (see to FIG. 2). The DSP server 25provides DSP resource management and telecommunications signalprocessing. Operating below the DSP server layer are a number ofphysical devices (PXD) 30 a, 30 b, 30 c. Each PXD provides an interfacebetween the DSP server 25 and an external telephony device (not shown)via a hardware abstraction layer (HAL) 34.

The DSP server 25 includes a resource manager 24 which receives commandsfrom, forwards events to, and exchanges data with the user applicationlayer 26. The user application layer 26 can either be resident on theDSP 17 or alternatively on the host 21 (see FIG. 2), such as amicrocontroller. An application programming interface 27 (API) providesa software interface between the user application layer 26 and theresource manager 24. The resource manager 24 manages theinternal/external program and data memory of the DSP 17. In addition theresource manager dynamically allocates DSP resources, performs commandrouting as well as other general purpose functions.

The DSP server 25 also includes virtual device drivers (VHDs) 22 a, 22b, 22 c. The VHDs are a collection of software objects that control theoperation of and provide the facility for real time signal processing.Each VHD 22 a, 22 b, 22 c includes an inbound and outbound media queue(not shown) and a library of signal processing services specific to thatVHD 22 a, 22 b, 22 c. In the described exemplary embodiment, each VHD 22a, 22 b, 22 c is a complete self-contained software module forprocessing a single channel with a number of different telephonydevices. Multiple channel capability can be achieved by adding VHDs tothe DSP server 25. The resource manager 24 dynamically controls thecreation and deletion of VHDs and services.

A switchboard 32 in the DSP server 25 dynamically inter-connects thePXDs 30 a, 30 b, 30 c with the VHDs 22 a, 22 b, 22 c. Each PXD 30 a, 30b, 30 c is a collection of software objects which provide signalconditioning for one external telephony device. For example, a PXD mayprovide volume and gain control for signals from a telephony deviceprior to communication with the switchboard 32. Multiple telephonyfunctionalities can be supported on a single channel by connectingmultiple PXDs, one for each telephony device, to a single VHD via theswitchboard 32. Connections within the switchboard 32 are managed by theuser application layer 26 via a set of API commands to the resourcemanager 24. The number of PXDs and VHDs is expandable, and limited onlyby the memory size and the MIPS (millions instructions per second) ofthe underlying hardware.

A hardware abstraction layer (HAL) 34 interfaces directly with theunderlying DSP 17 hardware (see FIG. 2) and exchanges telephony signalsbetween the external telephony devices and the PXDs. The HAL 34 includesbasic hardware interface routines, including DSP initialization, targethardware control, codec sampling, and hardware control interfaceroutines. The DSP initialization routine is invoked by the userapplication layer 26 to initiate the initialization of the signalprocessing system. The DSP initialization sets up the internal registersof the signal processing system for memory organization, interrupthandling, timer initialization, and DSP configuration. Target hardwareinitialization involves the initialization of all hardware devices andcircuits external to the signal processing system. The HAL 34 is aphysical firmware layer that isolates the communications software fromthe underlying hardware. This methodology allows the communicationssoftware to be ported to various hardware platforms by porting only theaffected portions of the HAL 34 to the target hardware.

The exemplary software architecture described above can be integratedinto numerous telecommunications products. In an exemplary embodiment,the software architecture is designed to support telephony signalsbetween telephony devices (and/or circuit switched networks) and packetbased networks. A network VHD (NetVHD) is used to provide a singlechannel of operation and provide the signal processing services fortransparently managing voice, fax, and modem data across a variety ofpacket based networks. More particularly, the NetVHD encodes andpacketizes DTMF, voice, fax, and modem data received from varioustelephony devices and/or circuit switched networks and transmits thepackets to the user application layer. In addition, the NetVHDdisassembles DTMF, voice, fax, and modem data from the user applicationlayer, decodes the packets into signals, and transmits the signals tothe circuit switched network or device.

An exemplary embodiment of the NetVHD operating in the describedsoftware architecture is shown in FIG. 4. The NetVHD includes fouroperational modes, namely voice mode 36, voiceband data mode 37, faxrelay mode 40, and data relay mode 42. In each operational mode, theresource manager invokes various services. For example, in the voicemode 36, the resource manager invokes call discrimination 44, packetvoice exchange 48, and packet tone exchange 50. The packet voiceexchange 48 may employ numerous voice compression algorithms, including,among others, Linear 128 kbps, G.711 u-law/A-law 64 kbps (ITURecommendation G.711 (1988)—Pulse code modulation (PCM) of voicefrequencies), G.726 16/24/32/40 kbps (ITU Recommendation G.726(12/90)—40, 32, 24, 16 kbits Adaptive Differential Pulse Code Modulation(ADPCM)), G.729A 8 kbps (Annex A (11/96) to ITU RecommendationG.729—Coding of speech at 8 kbit/s using conjugate structurealgebraic-code-excited linear-prediction (CS-ACELP)—Annex A: Reducedcomplexity 8 kbit/s CS-ACELP speech codec), and G.723 5.3/6.3 kbps (ITURecommendation G.723.1(03/96)—Dual rate coder for multimediacommunications transmitting at 5.3 and 6.3 kbit/s). The contents of eachof the foregoing ITU Recommendations being incorporated herein byreference as if set forth in full.

The packet voice exchange 48 is common to both the voice mode 36 and thevoiceband data mode 37. In the voiceband data mode 37, the resourcemanager invokes the packet voice exchange 48 for exchangingtransparently data without modification (other than packetization)between the telephony device (or circuit switched network) and thepacket based network. This is typically used for the exchange of fax andmodem data when bandwidth concerns are minimal as an alternative todemodulation and remodulation. During the voiceband data mode 37, thehuman speech detector service 59 is also invoked by the resourcemanager. The human speech detector 59 monitors the signal from the nearend telephony device for speech. In the event that speech is detected bythe human speech detector 59, an event is forwarded to the resourcemanager which, in turn, causes the resource manager to terminate thehuman speech detector service 59 and invoke the appropriate services forthe voice mode 36 (i.e., the call discriminator, the packet toneexchange, and the packet voice exchange).

In the fax relay mode 40, the resource manager invokes a fax exchange 52service. The packet fax exchange 52 may employ various data pumpsincluding, among others, V.17 which can operate up to 14,400 bits persecond, V.29 which uses a 1700-Hz caner that is varied in both phase andamplitude, resulting in 16 combinations of 8 phases and 4 amplitudeswhich can operate up to 9600 bits per second, and V.27ter which canoperate up to 4800 bits per second. Likewise, the resource managerinvokes a packet data exchange 54 service in the data relay mode 42. Thepacket data exchange 52 may employ various data pumps including, amongothers, V.22bis/V.22 with data rates up to 2400 bits per second,V.32bis/V.32 which enables full-duplex transmission at 14,400 bits persecond, and V.34 which operates up to 33,600 bits per second. The ITURecommendations setting forth the standards for the foregoing data pumpsare incorporated herein by reference as if set forth in full.

In the described exemplary embodiment, the user application layer doesnot need to manage any service directly. The user application layermanages the session using high-level commands directed to the NetVHD,which in turn directly runs the services. However, the user applicationlayer can access more detailed parameters of any service if necessary tochange, by way of example, default functions for any particularapplication.

In operation, the user application layer opens the NetVHD and connectsit to the appropriate PXD. The user application then may configurevarious operational parameters of the NetVHD, including, among others,default voice compression (Linear, G.711, G.726, G.723.1, G.723.1A,G.729A, G.729B), fax data pump (Binary, V.17, V.29, V.27ter), and modemdata pump (Binary, V.22bis, V.32bis, V.34). The user application layerthen loads an appropriate signaling service (not shown) into the NetVHD,configures it and sets the NetVHD to the On-hook state.

In response to events from the signaling service (not shown) via a nearend telephony device (hookswitch), or signal packets from the far end,the user application will set the NetVHD to the appropriate off-hookstate, typically voice mode. In an exemplary embodiment, if thesignaling service event is triggered by the near end telephony device,the packet tone exchange will generate dial tone. Once a DTMF tone isdetected, the dial tone is terminated. The DTMF tones are packetized andforwarded to the user application layer for transmission on the packetbased network. The packet tone exchange could also play ringing toneback to the near end telephony device (when a far end telephony deviceis being rung), and a busy tone if the far end telephony device isunavailable. Other tones may also be supported to indicate all circuitsare busy, or an invalid sequence of DTMF digits were entered on the nearend telephony device.

Once a connection is made between the near end and far end telephonydevices, the call discriminator is responsible for differentiatingbetween a voice and machine call by detecting the presence of a 2100 Hz.tone (as in the case when the telephony device is a fax or a modem), a1100 Hz. tone or V.21 modulated high level data link control (HDLC)flags (as in the case when the telephony device is a fax). If a 1100 Hz.tone, or V.21 modulated HDLC flags are detected, a calling fax machineis recognized. The NetVHD then terminates the voice mode 36 and invokesthe packet fax exchange to process the call. If however, 2100 Hz tone isdetected, the NetVHD terminates voice mode and invokes the packet dataexchange.

The packet data exchange service further differentiates between a faxand modem by continuing to monitor the incoming signal for V.21modulated HDLC flags, which if present, indicate that a fax connectionis in progress. If HDLC flags are detected, the NetVHD terminates packetdata exchange service and initiates packet fax exchange service.Otherwise, the packet data exchange service remains operative. In theabsence of an 1100 or 2100 Hz. tone, or V.21 modulated HDLC flags thevoice mode remains operative.

A. The Voice Mode

Voice mode provides signal processing of voice signals. As shown in theexemplary embodiment depicted in FIG. 5, voice mode enables thetransmission of voice over a packet based system such as Voice over IP(VoIP, H.323), Voice over Frame Relay (VoFR, FRF-11), Voice Telephonyover ATM (VTOA), or any other proprietary network. The voice mode shouldalso permit voice to be carried over traditional media such as timedivision multiplex (TDM) networks and voice storage and playbacksystems. Network gateway 55 a supports the exchange of voice between atraditional circuit switched 58 and a packet based network 56. Inaddition, network gateways 55 b, 55 c, 55 d, 55 e support the exchangeof voice between the packet based network 56 and a number of telephones57 a, 57 b, 57 c, 57 d, 57 e. Although the described exemplaryembodiment is shown for telephone communications across the packet basednetwork, it will be appreciated by those skilled in the art that othertelephony/network devices could be used in place of one or more of thetelephones, such as a HPNA phone connected via a cable modem.

The PXDs for the voice mode provide echo cancellation, gain, andautomatic gain control. The network VHD invokes numerous services in thevoice mode including call discrimination, packet voice exchange, andpacket tone exchange. These network VHD services operate together toprovide: (1) an encoder system with DTMF detection, call progress tonedetection, voice activity detection, voice compression, and comfortnoise estimation, and (2) a decoder system with delay compensation,voice decoding, DTMF generation, comfort noise generation and lost framerecovery.

The services invoked by the network VHD in the voice mode and theassociated PXD is shown schematically in FIG. 6. In the describedexemplary embodiment, the PXD 60 provides two way communication with atelephone or a circuit switched network, such as a PSTN line (e.g. DS0)carrying a 64 kb/s pulse code modulated (PCM) signal, i.e., digitalvoice samples.

The incoming PCM signal 60 a is initially processed by the PXD 60 toremove far end echos. As the name implies, echos in telephone systems isthe return of the talker's voice resulting from the operation of thehybrid with its two-four wire conversion. If there is low end-to-enddelay, echo from the far end is equivalent to side-tone (echo from thenear-end), and therefore, not a problem. Side-tone gives users feedbackas to how loud they are talking, and indeed, without side-tone, userstend to talk too loud. However, far end echo delays of more than about10 to 30 msec significantly degrade the voice quality and are a majorannoyance to the user.

An echo canceller 70 is used to remove echos from far end speech presenton the incoming PCM signal 60 a before routing the incoming PCM signal60 a back to the far end user. The echo canceller 70 samples an outgoingPCM signal 60 b from the far end user, filters it, and combines it withthe incoming PCM signal 60 a. Preferably, the echo canceller 70 isfollowed by a non-linear processor (NLP) 72 which may mute the digitalvoice samples when far end speech is detected in the absence of near endspeech. The echo canceller 70 may also inject comfort noise which in theabsence of near end speech may be roughly at the same level as the truebackground noise or at a fixed level.

After echo cancellation, the power level of the digital voice samples isnormalized by an automatic gain control (AGC) 74 to ensure that theconversation is of an acceptable loudness. Alternatively, the AGC can beperformed before the echo canceller 70, however, this approach wouldentail a more complex design because the gain would also have to beapplied to the sampled outgoing PCM signal 60 b. In the describedexemplary embodiment, the AGC 74 is designed to adapt slowly, althoughit should adapt fairly quickly if overflow or clipping is detected. TheAGC adaptation should be held fixed if the NLP 72 is activated.

After AGC, the digital voice samples are placed in the media queue 66 inthe network VHD 62 via the switchboard 32′. In the voice mode, thenetwork VHD 62 invokes three services, namely call discrimination,packet voice exchange, and packet tone exchange. The call discriminator68 analyzes the digital voice samples from the media queue to determinewhether a 2100 Hz, a 1100 Hz. tone or V.21 modulated HDLC flags arepresent. As described above with reference to FIG. 4, if either tone orHDLC flags are detected, the voice mode services are terminated and theappropriate service for fax or modem operation is initiated. In theabsence of a 2100 Hz, a 1100 Hz. tone, or HDLC flags, the digital voicesamples are coupled to the encoder system which includes a voice encoder82, a voice activity detector (VAD) 80, a comfort noise estimator 81, aDTMF detector 76, a call progress tone detector 77 and a packetizationengine 78.

Typical telephone conversations have as much as sixty percent silence orinactive content. Therefore, high bandwidth gains can be realized ifdigital voice samples are suppressed during these periods. A VAD 80,operating under the packet voice exchange, is used to accomplish thisfunction. The VAD 80 attempts to detect digital voice samples that donot contain active speech. During periods of inactive speech, thecomfort noise estimator 81 couples silence identifier (SID) packets to apacketization engine 78. The SID packets contain voice parameters thatallow the reconstruction of the background noise at the far end.

From a system point of view, the VAD 80 may be sensitive to the changein the NLP 72. For example, when the NLP 72 is activated, the VAD 80 mayimmediately declare that voice is inactive. In that instance, the VAD 80may have problems tracking the true background noise level. If the echocanceller 70 generates comfort noise during periods of inactive speech,it may have a different spectral characteristic from the true backgroundnoise. The VAD 80 may detect a change in noise character when the NLP 72is activated (or deactivated) and declare the comfort noise as activespeech. For these reasons, the VAD 80 should be disabled when the NLP 72is activated. This is accomplished by a “NLP on” message 72 a passedfrom the NLP 72 to the VAD 80.

The voice encoder 82, operating under the packet voice exchange, can bea straight 16 bit PCM encoder or any voice encoder which supports one ormore of the standards promulgated by ITU. The encoded digital voicesamples are formatted into a voice packet (or packets) by thepacketization engine 78. These voice packets are formatted according toan applications protocol and outputted to the host (not shown). Thevoice encoder 82 is invoked only when digital voice samples with speechare detected by the VAD 80. Since the packetization interval may be amultiple of an encoding interval, both the VAD 80 and the packetizationengine 78 should cooperate to decide whether or not the voice encoder 82is invoked. For example, if the packetization interval is 10 msec andthe encoder interval is 5 msec (a frame of digital voice samples is 5ms), then a frame containing active speech should cause the subsequentframe to be placed in the 10 ms packet regardless of the VAD stateduring that subsequent frame. This interaction can be accomplished bythe VAD 80 passing an “active” flag 80 a to the packetization engine 78,and the packetization engine 78 controlling whether or not the voiceencoder 82 is invoked.

In the described exemplary embodiment, the VAD 80 is applied after theAGC 74. This approach provides optimal flexibility because both the VAD80 and the voice encoder 82 are integrated into some speech compressionschemes such as those promulgated in ITU Recommendations G.729 withAnnex B VAD (March 1996)—Coding of Speech at 8 kbits/s UsingConjugate-Structure Algebraic-Code-Exited Linear Prediction (CS-ACELP),and G.723.1 with Annex A VAD (March 1996)—Dual Rate Coder for MultimediaCommunications Transmitting at 5.3 and 6.3 kbit/s, the contents of whichis hereby incorporated by reference as through set forth in full herein.

Operating under the packet tone exchange, a DTMF detector 76 determineswhether or not there is a DTMF signal present at the near end. The DTMFdetector 76 also provides a pre-detection flag 76 a which indicateswhether or not it is likely that the digital voice sample might be aportion of a DTMF signal. If so, the pre-detection flag 76 a is relayedto the packetization engine 78 instructing it to begin holding voicepackets. If the DTMF detector 76 ultimately detects a DTMF signal, thevoice packets are discarded, and the DTMF signal is coupled to thepacketization engine 78. Otherwise the voice packets are ultimatelyreleased from the packetization engine 78 to the host (not shown). Thebenefit of this method is that there is only a temporary impact on voicepacket delay when a DTMF signal is pre-detected in error, and not aconstant buffering delay. Whether voice packets are held while thepre-detection flag 76 a is active could be adaptively controlled by theuser application layer.

Similarly, a call progress tone detector 77 also operates under thepacket tone exchange to determine whether a precise signaling tone ispresent at the near end. Call progress tones are those which indicatewhat is happening to dialed phone calls. Conditions like busy line,ringing called party, bad number, and others each have distinctive tonefrequencies and cadences assigned them. The call progress tone detector77 monitors the call progress state, and forwards a call progress tonesignal to the packetization engine to be packetized and transmittedacross the packet based network. The call progress tone detector mayalso provide information regarding the near end hook status which isrelevant to the signal processing tasks. If the hook status is on hook,the VAD should preferably mark all frames as inactive, DTMF detectionshould be disabled, and SID packets should only be transferred if theyare required to keep the connection alive.

The decoding system of the network VHD 62 essentially performs theinverse operation of the encoding system. The decoding system of thenetwork VHD 62 comprises a depacketizing engine 84, a voice queue 86, aDTMF queue 88, a precision tone queue 87, a voice synchronizer 90, aDTMF synchronizer 102, a precision tone synchronizer 103, a voicedecoder 96, a VAD 98, a comfort noise estimator 100, a comfort noisegenerator 92, a lost packet recovery engine 94, a tone generator 104,and a precision tone generator 105.

The depacketizing engine 84 identifies the type of packets received fromthe host (i.e., voice packet, DTMF packet, call progress tone packet,SID packet), transforms them into frames which are protocol independent.The depacketizing engine 84 then transfers the voice frames (or voiceparameters in the case of SID packets) into the voice queue 86,transfers the DTMF frames into the DTMF queue 88 and transfers the callprogress tones into the call progress tone queue 87. In this manner, theremaining tasks are, by and large, protocol independent.

A jitter buffer is utilized to compensate for network impairments suchas delay jitter caused by packets not arriving at the same time or inthe same order in which they were transmitted. In addition, the jitterbuffer compensates for lost packets that occur on occasion when thenetwork is heavily congested. In the described exemplary embodiment, thejitter buffer for voice includes a voice synchronizer 90 that operatesin conjunction with a voice queue 86 to provide an isochronous stream ofvoice frames to the voice decoder 96.

Sequence numbers embedded into the voice packets at the far end can beused to detect lost packets, packets arriving out of order, and shortsilence periods. The voice synchronizer 90 can analyze the sequencenumbers, enabling the comfort noise generator 92 during short silenceperiods and performing voice frame repeats via the lost packet recoveryengine 94 when voice packets are lost. SID packets can also be used asan indicator of silent periods causing the voice synchronizer 90 toenable the comfort noise generator 92. Otherwise, during far end activespeech, the voice synchronizer 90 couples voice frames from the voicequeue 86 in an isochronous stream to the voice decoder 96. The voicedecoder 96 decodes the voice frames into digital voice samples suitablefor transmission on a circuit switched network, such as a 64 kb/s PCMsignal for a PSTN line. The output of the voice decoder 96 (or thecomfort noise generator 92 or lost packet recovery engine 94 if enabled)is written into a media queue 106 for transmission to the PXD 60.

The comfort noise generator 92 provides background noise to the near enduser during silent periods. If the protocol supports SID packets, (andthese are supported for VTOA, FRF-11, and VoIP), the comfort noiseestimator at the far end encoding system should transmit SID packets.Then, the background noise can be reconstructed by the near end comfortnoise generator 92 from the voice parameters in the SID packets bufferedin the voice queue 86. However, for some protocols, namely, FRF-11, theSID packets are optional, and other far end users may not support SIDpackets at all. In these systems, the voice synchronizer 90 mustcontinue to operate properly. In the absence of SID packets, the voiceparameters of the background noise at the far end can be determined byrunning the VAD 98 at the voice decoder 96 in series with a comfortnoise estimator 100.

Preferably, the voice synchronizer 90 is not dependent upon sequencenumbers embedded in the voice packet. The voice synchronizer 90 caninvoke a number of mechanisms to compensate for delay jitter in thesesystems. For example, the voice synchronizer 90 can assume that thevoice queue 86 is in an underflow condition due to excess jitter andperform packet repeats by enabling the lost frame recovery engine 94.Alternatively, the VAD 98 at the voice decoder 96 can be used toestimate whether or not the underflow of the voice queue 86 was due tothe onset of a silence period or due to packet loss. In this instance,the spectrum and/or the energy of the digital voice samples can beestimated and the result 98 a fed back to the voice synchronizer 90. Thevoice synchronizer 90 can then invoke the lost packet recovery engine 94during voice packet losses and the comfort noise generator 92 duringsilent periods.

When DTMF packets arrive, they are depacketized by the depacketizingengine 84. DTMF frames at the output of the depacketizing engine 84 arewritten into the DTMF queue 88. The DTMF synchronizer 102 couples theDTMF frames from the DTMF queue 88 to the tone generator 104. Much likethe voice synchronizer, the DTMF synchronizer 102 is employed to providean isochronous stream of DTMF frames to the tone generator 104.Generally speaking, when DTMF packets are being transferred, voiceframes should be suppressed. To some extent, this is protocol dependent.However, the capability to flush the voice queue 86 to ensure that thevoice frames do not interfere with DTMF generation is desirable.Essentially, old voice frames which may be queued are discarded whenDTMF packets arrive. This will ensure that there is a significantinter-digit gap before DTMF tones are generated. This is achieved by a“tone present” message 88 a passed between the DTMF queue and the voicesynchronizer 90.

The tone generator 104 converts the DTMF signals into a DTMF tonesuitable for a standard digital or analog telephone. The tone generator104 overwrites the media queue 106 to prevent leakage through the voicepath and to ensure that the DTMF tones are not too noisy.

There is also a possibility that DTMF tone may be fed back as an echointo the DTMF detector 76. To prevent false detection, the DTMF detector76 can be disabled entirely (or disabled only for the digit beinggenerated) during DTMF tone generation. This is achieved by a “tone on”message 104 a passed between the tone generator 104 and the DTMFdetector 76. Alternatively, the NLP 72 can be activated while generatingDTMF tones.

When call progress tone packets arrive, they are depacketized by thedepacketizing engine 84. Call progress tone frames at the output of thedepacketizing engine 84 are written into the call progress tone queue87. The call progress tone synchronizer 103 couples the call progresstone frames from the call progress tone queue 87 to a call progress tonegenerator 105. Much like the DTMF synchronizer, the call progress tonesynchronizer 103 is employed to provide an isochronous stream of callprogress tone frames to the call progress tone generator 105. And muchlike the DTMF tone generator, when call progress tone packets are beingtransferred, voice frames should be suppressed. To some extent, this isprotocol dependent. However, the capability to flush the voice queue 86to ensure that the voice frames do not interfere with call progress tonegeneration is desirable. Essentially, old voice frames which may bequeued are discarded when call progress tone packets arrive to ensurethat there is a significant inter-digit gap before call progress tonesare generated. This is achieved by a “tone present” message 87 a passedbetween the call progress tone queue 87 and the voice synchronizer 90.

The call progress tone generator 105 converts the call progress tonesignals into a call progress tone suitable for a standard digital oranalog telephone. The call progress tone generator 105 overwrites themedia queue 106 to prevent leakage through the voice path and to ensurethat the call progress tones are not too noisy.

The outgoing PCM signal in the media queue 106 is coupled to the PXD 60via the switchboard 32′. The outgoing PCM signal is coupled to anamplifier 108 before being outputted on the PCM output line 60 b.

1. Echo Canceller with NLP

The problem of line echos such as the reflection of the talker's voiceresulting from the operation of the hybrid with its two-four wireconversion is a common telephony problem. To eliminate or minimize theeffect of line echos in the described exemplary embodiment of thepresent invention, an echo canceller with non-linear processing is used.Although echo cancellation is described in the context of a signalprocessing system for packet voice exchange, those skilled in the artwill appreciate that the techniques described for echo cancellation arelikewise suitable for various applications requiring the cancellation ofreflections, or other undesirable signals, from a transmission line.Accordingly, the described exemplary embodiment for echo cancellation ina signal processing system is by way of example only and not by way oflimitation.

In the described exemplary embodiment the echo canceller preferablycomplies with one or more of the following ITU-T Recommendations G.164(1988)—Echo Suppressors, G.165 (March 1993)—Echo Cancellers, and G.168(April 1997)—Digital Network Echo Cancellers, the contents of which areincorporated herein by reference as though set forth in full. Thedescribed embodiment merges echo cancellation and echo suppressionmethodologies to remove the line echos that are prevalent intelecommunication systems. Typically, echo cancellers are favored overecho suppressors for superior overall performance in the presence ofsystem noise such as, for example, background music, double talk etc.,while echo suppressors tend to perform well over a wide range ofoperating conditions where clutter such as system noise is not present.The described exemplary embodiment utilizes an echo suppressor when theenergy level of the line echo is below the audible threshold, otherwisean echo canceller is preferably used. The use of an echo suppressorreduces system complexity, leading to lower overall power consumption orhigher densities (more VHDs per part or network gateway). Those skilledin the art will appreciate that various signal characteristics such asenergy, average magnitude, echo characteristics, as well as informationexplicitly received in voice or SID packets may be used to determinewhen to bypass echo cancellation. Accordingly, the described exemplaryembodiment for bypassing echo cancellation in a signal processing systemas a function of estimated echo power is by way of example only and notby way of limitation.

FIG. 7 shows the block diagram of an echo canceller in accordance with apreferred embodiment of the present invention. If required to supportvoice transmission via a T1 or other similar transmission media, acompressor 120 may compress the output 120(a) of the voice decodersystem into a format suitable for the channel at R_(out) 120(b).Typically the compressor 120 provides Haw or μ-law compression inaccordance with ITU-T standard G.711, although linear compression orcompression in accordance with alternate companding laws may also besupported. The compressed signal at R_(out) (signal that eventuallymakes it way to a near end ear piece/telephone receiver), may bereflected back as an input signal to the voice encoder system. An inputsignal 122(a) may also be in the compressed domain (if compressed bycompressor 120) and, if so, an expander 122 may be required to invertthe companding law to obtain a near end signal 122(b). A power estimator124 estimates a short term average power 124(a), a long term averagepower 124(b), and a maximum power level 124(c) for the near end signal122(b).

An expander 126 inverts the companding law used to compress the voicedecoder output signal 120(b) to obtain a reference signal 126(a). One ofskill in the art will appreciated that the voice decoder output signalcould alternatively be compressed downstream of the echo canceller sothat the expander 126 would not be required. However, to ensure that allnon-linearities in the echo path are accounted for in the referencesignal 126(a) it is preferable to compress/expand the voice decoderoutput signal 120(b). A power estimator 128 estimates a short termaverage power 128(a), a long term average power 128(b), a maximum powerlevel 128(c) and a background power level 128(d) for the referencesignal 126(a). The reference signal 126(a) is input into a finiteimpulse response (FIR) filter 130. The FIR filter 130 models thetransfer characteristics of a dialed telephone line circuit so that theunwanted echo may preferably be canceled by subtracting filteredreference signal 130(a) from the near end signal 122(b) in a differenceoperator 132.

However, for a variety of reasons, such as for example, non-linearitiesin the hybrid and tail circuit, estimation errors, noise in the system,etc., the adaptive FIR filter 130 may not identically model the transfercharacteristics of the telephone line circuit so that the echo cancellermay be unable to cancel all of the resulting echo. Therefore, a nonlinear processor (NLP) 140 is used to suppress the residual echo duringperiods of far end active speech with no near end speech. During periodsof inactive speech, a power estimator 138 estimates the performance ofthe echo canceller by estimating a short term average power 138(a), along term average power 138(b) and background power level 138(c) for anerror signal 132(b) which is an output of the difference operator 132.The estimated performance of the echo canceller is one measure utilizedby adaptation logic 136 to selectively enable a filter adapter 134 whichcontrols the convergence of the adaptive FIR filter 130. The adaptationlogic 136 processes the estimated power levels of the reference signal(128 a, 128 b, 128 c and 128 d) the near end signal (124 a, 124 b and124 c) and the error signal (138 a, 138 b and 138 c) to control theinvocation of the filter adapter 134 as well as the step size to be usedduring adaptation.

In the described preferred embodiment, the echo suppressor is a simplebypass 144(a) that is selectively enabled by toggling the bypasscancellation switch 144. A bypass estimator 142 toggles the bypasscancellation switch 144 based upon the maximum power level 128(c) of thereference signal 126(a), the long term average power 138(b) of the errorsignal 132(b) and the long term average power 124(b) of the near endsignal 122(b). One skilled in the art will appreciate that a NLP orother suppressor could be included in the bypass path 144(a), so thatthe described echo suppressor is by way of example only and not by wayof limitation.

In an exemplary embodiment, the adaptive filter 130 models the transfercharacteristics of the hybrid and the tail circuit of the telephonecircuit. The tail length supported should preferably be at least 16msec. The adaptive filter 130 may be a linear transversal filter orother suitable finite impulse response filter. In the describedexemplary embodiment, the echo canceller preferably converges or adaptsonly in the absence of near end speech. Therefore, near end speechand/or noise present on the input signal 122(a) may cause the filteradapter 134 to diverge. To avoid divergence the filter adapter 134 ispreferably selectively enabled by the adaptation logic 136. In addition,the time required for an adaptive filter to converge increasessignificantly with the number of coefficients to be determined.Reasonable modeling of the hybrid and tail circuits with a finiteimpulse response filter requires a large number of coefficients so thatfilter adaptation is typically computationally intense. In the describedexemplary embodiment the DSP resources required for filter adaptationare minimized by adjusting the adaptation speed of the FIR filter 130.

The filter adapter 134 is preferably based upon a normalized least meansquare algorithm (NLMS) as described in S. Haykin, Adaptive FilterTheory, and T. Parsons, Voice and Speech Processing, the contents ofwhich are incorporated herein by reference as if set forth in full. Theerror signal 132(b) at the output of the difference operator 132 for theadaptation logic may preferably be characterized as follows:

${e(n)} = {{s(n)} - {\sum\limits_{j = 0}^{L - 1}{{c(j)}{r\left( {n - j} \right)}}}}$

where e(n) is the error signal at time n, r(n) is the reference signal126(a) at time n and s(n) is the near end signal 122(b) at time n, andc(j) are the coefficients of the transversal filter where the dimensionof the transversal filter is preferably the worst case echo path length(i.e. the length of the tail circuit L) and c(j), for j=0 to L−1, isgiven by:

c(j)=c(j)+μ*e(n)*r(n−j)

wherein c(j) is preferably initialized to a reasonable value such as forexample zero.

Assuming a block size of one msec (or 8 samples at a sampling rate of 8kHz), the short term average power of the reference signal P_(ref) isthe sum of the last L reference samples and the energy for the currenteight samples so that

$\mu = \frac{\alpha}{P_{{ref}{(n)}}}$

where α is the adaptation step size. One of skill in the art willappreciate that the filter adaptation logic may be implemented in avariety of ways, including fixed point rather than the describedfloating point realization. Accordingly, the described exemplaryadaptation logic is by way of example only and not by way of limitation.

To support filter adaptation the described exemplary embodiment includesthe power estimator 128 that estimates the short term average power128(a) of the reference signal 126(a) (P_(ref)). In the describedexemplary embodiment the short term average power is preferablyestimated over the worst case length of the echo path plus eightsamples, (i.e. the length of the FIR filter L+8 samples). In addition,the power estimator 128 computes the maximum power level 128(c) of thereference signal 126(a) (P_(refmax)) over a period of time that ispreferably equal to the tail length L of the echo path. For example,putting a time index on the short term average power, so that P_(ref)(n)is the power of the reference signal at time n. P_(refmax) is thencharacterized as:

P _(refmax)(n)=max P _(ref)(j) for j=n−L msec to j=n

where Lmsec is the length of the tail in msec so that P_(refmax) is themaximum power in the reference signal P_(ref) over a length of timeequal to the tail length.

The second power estimator 124 estimates the short term average power ofthe near end signal 122(b) (P_(near)) in a similar manner. The shortterm average power 138(a) of the error signal 132(b) (the output ofdifference operator 132), P_(err) is also estimated in a similar mannerby the third power estimator 138.

In addition, the echo return loss (ERL), defined as the loss fromR_(out) 120(b) to S_(in) 122(a) in the absence of near end speech, isperiodically estimated and updated. In the described exemplaryembodiment the ERL is estimated and updated about every 5-20 msec. Thepower estimator 128 estimates the long term average power 128(b)(P_(refERL)) of the reference signal 126(a) in the absence of near endspeech. The second power estimator 124 estimates the long term averagepower 124(b) (P_(nearERL)) of the near end signal 122(b) in the absenceof near end speech. The adaptation logic 136 computes the ERL bydividing the long term average power of the reference signal(P_(refERL)) by the long term average power of the near end signal(P_(nearERL)). The adaptation logic 136 preferably only updates the longterm averages used to compute the estimated ERL if the estimated shortterm power level 128(a) (P_(ref)) of the reference signal 126(a) isgreater than a predetermined threshold, preferably in the range of about−30 to −35 dBmO; and the estimated short term power level 128(a)(P_(ref)) of the reference signal 126(a) is preferably larger than aboutat least the short term average power 124(a) (P_(near)) of the near endsignal 122(b) (P_(ref)>P_(near) in the preferred embodiment).

In the preferred embodiment, the long term averages (P_(refERL) andP_(nearERL)) are based on a first order infinite impulse response (IIR)recursive filter, wherein the inputs to the two first order filters areP_(ref) and P_(near).

P _(nearERL)=(1-beta)*P _(nearERL) +P _(near)* beta; and

P_(refERL)=(1-beta)*P _(refERL) +P _(ref)*beta

where filter coefficient beta= 1/64

Similarly, the adaptation logic 136 of the described exemplaryembodiment characterizes the effectiveness of the echo canceller byestimating the echo return loss enhancement (ERLE). The ERLE is anestimation of the reduction in power of the near end signal 122(b) dueto echo cancellation when there is no near end speech present. The ERLEis the average loss from the input 132(a) of the difference operator 132to the output 132(b) of the difference operator 132. The adaptationlogic 136 in the described exemplary embodiment periodically estimatesand updates the ERLE, preferably in the range of about 5 to 20 msec. Inoperation, the power estimator 124 estimates the long term average power124(b) P_(nearERLE) of the near end signal 122(b) in the absence of nearend speech. The power estimator 138 estimates the long term averagepower 138(b) P_(errERLE) of the error signal 132(b) in the absence ofnear end speech. The adaptation logic 136 computes the ERLE by dividingthe long term average power 124(a) P_(nearERLE) of the near end signal122(b) by the long term average power 138(b) P_(errERLE) of the errorsignal 132(b). The adaptation logic 136 preferably updates the long termaverages used to compute the estimated ERLE only when the estimatedshort term average power 128(a) (P_(ref)) of the reference signal 126(a)is greater than a predetermined threshold preferably in the range ofabout −30 to −35 dBmO; and the estimated short term average power 124(a)(P_(near)) 1 of the near end signal 122(b) is large as compared to theestimated short term average power 138(a) (P_(err)) of the error signal(preferably when P_(near) is approximately greater than or equal to fourtimes the short term average power of the error signal (4P_(err))).Therefore, an ERLE of approximately 6 dB is preferably required beforethe ERLE tracker will begin to function.

In the preferred embodiment, the long term averages (P_(nearERLE) andP_(errERLE)) may be based on a first order IIR (infinite impulseresponse) recursive filter, wherein the inputs to the two first orderfilters are P_(near) and P_(err)

P _(nearERLE)=(1-beta)*P _(nearERL) +P _(near)*beta; and

P _(errERLE)=(1-beta)*P _(errERL) +P _(err)*beta

where filter coefficient beta= 1/64

It should be noted that PnearERL≠PnearERLE because the conditions underwhich each is updated are different.

To assist in the determination of whether to invoke the echo cancellerand if so with what step size, the described exemplary embodimentestimates the power level of the background noise. The power estimator128 tracks the long term energy level of the background noise 128(d)(B_(ref)) of the reference signal 126(a). The power estimator 128utilizes a much faster time constant when the input energy is lower thanthe background noise estimate (current output). With a fast timeconstant the power estimator 128 tends to track the minimum energy levelof the reference signal 126(a). By definition, this minimum energy levelis the energy level of the background noise of the reference signalB_(ref). The energy level of the background noise of the error signalB_(err) is calculated in a similar manner. The estimated energy level ofthe background noise of the error signal (B_(err)) is not updated whenthe energy level of the reference signal is larger than a predeterminedthreshold (preferably in the range of about 30-35 dBmO).

In addition, the invocation of the echo canceller depends on whethernear end speech is active. Preferably, the adaptation logic 136 declaresnear end speech active when three conditions are met. First, the shortterm average power of the error signal should preferably exceed aminimum threshold, preferably on the order of about −36 dBmO(P_(err)≳−36 dBmO). Second, the short term average power of the errorsignal should preferably exceed the estimated power level of thebackground noise for the error signal by preferably at least about 6 dB(P_(err)≳B_(err)+6 dB). Third, the short term average power 124(a) ofthe near end signal 122(b) is preferably approximately 3 dB greater thanthe maximum power level 128(c) of the reference signal 126(a) less theestimated ERL (P_(near)≳P_(refmax)−ERL+3 dB). The adaptation logic 136preferably sets a near end speech hangover counter (not shown) when nearend speech is detected. The hangover counter is used to prevent clippingof near end speech by delaying the invocation of the NLP 140 when nearend speech is detected. Preferably the hangover counter is on the orderof about 150 msec.

In the described exemplary embodiment, if the maximum power level(P_(refmax)) of the reference signal minus the estimated ERL is lessthan the threshold of hearing (all in dB) neither echo cancellation ornon-linear processing are invoked. In this instance, the energy level ofthe echo is below the threshold of hearing, typically about −65 to −69dBmO, so that echo cancellation and non-linear processing are notrequired for the current time period. Therefore, the bypass estimator142 sets the bypass cancellation switch 144 in the down position, so asto bypass the echo canceller and the NLP and no processing (other thanupdating the power estimates) is performed. Also, if the maximum powerlevel (P_(refmax)) of the reference signal minus the estimated ERL isless than the maximum of either the threshold of hearing, or backgroundpower level B_(err) of the error signal minus a predetermined threshold(P_(refmax)−ERL<threshold of hearing or (B_(err)−threshold)) neitherecho cancellation or non-linear processing are invoked. In thisinstance, the echo is buried in the background noise or below thethreshold of hearing, so that echo cancellation and non-linearprocessing are not required for the current time period. In thedescribed preferred embodiment the background noise estimate ispreferably greater than the threshold of hearing, such that this is abroader method for setting the bypass cancellation switch. The thresholdis preferably in the range of about 8-12 dB.

Similarly, if the maximum power level (P_(refmax)) of the referencesignal minus the estimated ERL is less than the short term average powerP_(near) minus a predetermined threshold(P_(refmax)−ERL<P_(near)−threshold) neither echo cancellation ornon-linear processing are invoked. In this instance, it is highlyprobable that near end speech is present, and that such speech willlikely mask the echo. This method operates in conjunction with the abovedescribed techniques for bypassing the echo canceller and NLP. Thethreshold is preferably in the range of about 8-12 dB. If the NLPcontains a real comfort noise generator, i.e., a non-linearity whichmutes the incoming signal and injects comfort noise of the appropriatecharacter then a determination that the NLP will be invoked in theabsence of filter adaptation allows the adaptive filter to be bypassedor not invoked. This method is, used in conjunction with the abovemethods. If the adaptive filter is not executed then adaptation does nottake place, so this method is preferably used only when the echocanceller has converged.

If the bypass cancellation switch 144 is in the down position, theadaptation logic 136 disables the filter adapter 134. Otherwise, forthose conditions where the bypass cancellation switch 144 is in the upposition so that both adaptation and cancellation may take place, theoperation of the preferred adaptation logic 136 proceeds as follows:

If the estimated echo return loss enhancement is low (preferably in therange of about 0-9 dBm) the adaptation logic 136 enables rapidconvergence with an adaptation step size α=¼. In this instance, the echocanceller is not converged so that rapid adaptation is warranted.However, if near end speech is detected within the hangover period, theadaptation logic 136 either disables adaptation or uses very slowadaptation, preferably an adaptation speed on the order of aboutone-eighth that used for rapid convergence or an adaptation step size α=1/32. In this case the adaptation logic 136 disables adaptation when theecho canceller is converged. Convergence may be assumed if adaptationhas been active for a total of one second after the off hook transitionor subsequent to the invocation of the echo canceller. Otherwise if thecombined loss (ERL+ERLE) is in the range of about 33-36 dB, theadaptation logic 136 enables slow adaptation (preferably one-eighth theadaptation speed of rapid convergence or an adaptation step size α=1/32). If the combined loss (ERL+ERLE) is in the range of about 23-33dB, the adaptation logic 136 enables a moderate convergence speed,preferably on the order of about one-fourth the adaptation speed usedfor rapid convergence or an adaptation step size α= 1/16.

Otherwise, one of three preferred adaptation speeds is chosen based onthe estimated echo power (P_(refmax) refmax minus the ERL) in relationto the power level of the background noise of the error signal. If theestimated echo power (P_(refmax)−ERL) is large compared to the powerlevel of the background noise of the error signal(P_(refmax)−ERL≧B_(err)+24 dB), rapid adaptation/convergence is enabledwith an adaptation step size on the order of about α=¼. Otherwise, if(P_(refmax)−ERL≳B_(err)+18 dB) the adaptation speed is reduced toapproximately one-half the adaptation speed used for rapid convergenceor an adaptation step size on the order of about α=⅛. Otherwise, if(P_(refmax)−ERL≳B_(err)+9 dB) the adaptation speed is further reduced toapproximately one-quarter the adaptation speed used for rapidconvergence or an adaptation step size α= 1/16.

As a further limit on adaptation speed, if echo canceller adaptation hasbeen active for a sum total of one second since initialization or anoff-hook condition then the maximum adaptation speed is limited toone-fourth the adaptation speed used for rapid convergence (α= 1/16).Also, if the echo path changes appreciably or if for any reason theestimated ERLE is negative, (which typically occurs when the echo pathchanges) then the coefficients are cleared and an adaptation counter isset to zero (the adaptation counter measures the sum total of adaptationcycles in samples).

The NLP 140 is a two state device. The NLP 140 is either on (applyingnon-linear processing) or it is off (applying unity gain). When the NLP140 is on it tends to stay on, and when the NLP 140 is off it tends tostay off. The NLP 140 is preferably invoked when the bypass cancellationswitch 144 is in the upper position so that adaptation and cancellationare active. Otherwise, the NLP 140 is not invoked and the NLP 140 isforced into the off state.

Initially, a stateless first NLP decision is created. The decision logicis based on three decision variables (D1-D3). The decision variable D1is set if it is likely that the far end is active (i.e. the short termaverage power 128(a) of the reference signal 126(a) is preferably about6 dB greater than the power level of the background noise 128(d) of thereference signal), and the short term average power 128(a) of thereference signal 126(a) minus the estimated ERL is greater than theestimated short term average power 124(a) of the near end signal 122(b)minus a small threshold, preferably in the range of about 6 dB. In thepreferred embodiment, this is represented by: (P_(ref)≳B_(ref)+6 dB) and((P_(ref)−ERL)≳(P_(near)−6 dB)). Thus, decision variable D1 attempts todetect far end active speech and high ERL (implying no near end).Preferably, decision variable D2 is set if the power level of the errorsignal is on the order of about 9 dB below the power level of theestimated short term average power 124(a) of the near end signal 122(b)(a condition that is indicative of good short term ERLE). In thepreferred embodiment, P_(err)≲P_(near)−9 dB is used (a short term ERLEof 9 dB). The third decision variable D3 is preferably set if thecombined loss (reference power to error power) is greater than athreshold. In the preferred embodiment, this is: P_(err)≲P_(ref)−t,where t is preferably initialized to about 6 dB and preferably increasesto about 12 dB after about one second of adaptation. (In other words, itis only adapted while convergence is enabled).

The third decision variable D3 results in more aggressive non linearprocessing while the echo canceller is uncoverged. Once the echocanceller converges, the NLP 140 can be slightly less aggressive. Theinitial stateless decision is set if two of the sub-decisions or controlvariables are initially set. The initial decision set implies that theNLP 140 is in a transition state or remaining on.

A NLP state machine (not shown) controls the invocation and terminationof NLP 140 in accordance with the detection of near end speech aspreviously described. The NLP state machine delays activation of the NLP140 when near end speech is detected to prevent clipping the near endspeech. In addition, the NLP state machine is sensitive to the near endspeech hangover counter (set by the adaptation logic when near endspeech is detected) so that activation of the NLP 140 is further delayeduntil the near end speech hangover counter is cleared. The NLP statemachine also deactivates the NLP 140. The NLP state machine preferablysets an off counter when the NLP 140 has been active for a predeterminedperiod of time, preferably about the tail length in msec. The “off”counter is cleared when near end speech is detected and decrementedwhile non-zero when the NLP is on. The off counter delays termination ofNLP processing when the far end power decreases so as to prevent thereflection of echo stored in the tail circuit. If the near end speechdetector hangover counter is on, the above NLP decision is overriden andthe NLP is forced into the off state.

In the preferred embodiment, the NLP 140 may be implemented with asuppressor that adaptively suppresses down to the background noise level(B_(err)), or a suppressor that suppresses completely and insertscomfort noise with a spectrum that models the true background noise.

2. Automatic Gain Control

In an exemplary embodiment of the present invention, AGC is used tonormalize digital voice samples to ensure that the conversation betweenthe near and far end users is maintained at an acceptable volume. Thedescribed exemplary embodiment of the AGC includes a signal bypass forthe digital voice samples when the gain adjusted digital samples exceedsa predetermined power level. This approach provides rapid response timeto increased power levels by coupling the digital voice samples directlyto the output of the AGC until the gain falls off due to AGC adaptation.Although AGC is described in the context of a signal processing systemfor packet voice exchange, those skilled in the art will appreciate thatthe techniques described for AGC are likewise suitable for variousapplications requiring a signal bypass when the processing of the signalproduces undesirable results. Accordingly, the described exemplaryembodiment for AGC in a signal processing system is by way of exampleonly and not by way of limitation.

In an exemplary embodiment, the AGC can be either fully adaptive or havea fixed gain. Preferably, the AGC supports a fully adaptive operatingmode with a range of about −30 dB to 30 dB. A default gain value may beindependently established, and is typically 0 dB. If adaptive gaincontrol is used, the initial gain value is specified by this defaultgain. The AGC adjusts the gain factor in accordance with the power levelof an input signal. Input signals with a low energy level are amplifiedto a comfortable sound level, while high energy signals are attenuated.

A block diagram of a preferred embodiment of the AGC is shown in FIG.8A. A multiplier 150 applies a gain factor 152 to an input signal 150(a)which is then output to the media queue 66 of the network VHD (see FIG.6). The default gain, typically 0 dB is initially applied to the inputsignal 150(a). A power estimator 154 estimates the short term averagepower 154(a) of the gain adjusted signal 150(b). The short term averagepower of the input signal 150(a) is preferably calculated every eightsamples, typically every one ms for a 8 kHz signal. Clipping logic 156analyzes the short term average power 154(a) to identify gain adjustedsignals 150(b) whose amplitudes are greater than a predeterminedclipping threshold. The clipping logic 156 controls an AGC bypass switch157, which directly connects the input signal 150(a) to the media queue66 when the amplitude of the gain adjusted signal 150(b) exceeds thepredetermined clipping threshold. The AGC bypass switch 157 remains inthe up or bypass position until the AGC adapts so that the amplitude ofthe gain adjusted signal 150(b) falls below the clipping threshold.

The power estimator 154 also calculates a long term average power 154(b)for the input signal 150(a), by averaging thirty two short term averagepower estimates, (i.e. averages thirty two blocks of eight samples). Thelong term average power is a moving average which provides significanthangover. A peak tracker 158 utilizes the long term average power 154(b)to calculate a reference value which gain calculator 160 utilizes toestimate the required adjustment to a gain factor 152. The gain factor152 is applied to the input signal 150(a) by the multiplier 150. In thedescribed exemplary embodiment the peak tracker 158 may preferably be anon-linear filter. The peak tracker 158 preferably stores a referencevalue which is dependent upon the last maximum peak. The peak tracker158 compares the long term average power estimate to the referencevalue. FIG. 8B shows the peak tracker output as a function of an inputsignal, demonstrating that the reference value that the peak tracker 158forwards to the gain calculator 160 should preferably rise quickly ifthe signal amplitude increases, but decrement slowly if the signalamplitude decreases. Thus for active voice segments followed by silence,the peak tracker output slowly decreases, so that the gain factorapplied to the input signal 150(a) may be slowly increased. However, forlong inactive or silent segments followed by loud or high amplitudevoice segments, the peak tracker output increases rapidly, so that thegain factor applied to the input signal 150(a) may be quickly decreased.

In the described exemplary embodiment, the peak tracker should beupdated when the estimated long term power exceeds the threshold ofhearing. Peak tracker inputs include the current estimated long termpower level a(i), the previous long term power estimate a(i-1) and theprevious peak tracker output x(i-1). In operation, when the long termenergy is varying rapidly, preferably when the previous long term powerestimate is on the order of four times greater than the current longterm estimate or vice versa, the peak tracker should go into hangovermode. In hangover mode, the peak tracker should not be updated. Thehangover mode prevents adaptation on impulse noise.

If the long term energy estimate is large compared to the previous peaktracker estimate, then the peak tracker should adapt rapidly. In thiscase the current peak tracker output x(i) is given by:

x(i)=(7x(i−1)+a(i))/8.

where x(i-1) is the previous peak tracker output and a(i) is the currentlong term power estimate.

If the long term energy is less than the previous peak tracker output,then the peak tracker will adapt slowly. In this case the current peaktracker output x(i) is given by:

x(i)=x(i−1)*255/256.

Referring to FIG. 9, a preferred embodiment of the gain calculator 160slowly increments the gain factor 152 for signals below the comfortlevel of hearing 166 (below minVoice) and decrements the gain forsignals above the comfort level of hearing 164 (above MaxVoice). Thedescribed exemplary embodiment of the gain calculator 160 decrements thegain factor 152 for signals above the clipping threshold relativelyfast, preferably on the order of about 2-4 dB/sec, until the signal hasbeen attenuated approximately 10 dB or the power level of the signaldrops to the comfort zone. The gain calculator 160 preferably decrementsthe gain factor 152 for signals with power levels that are above thecomfort level of hearing 164 (MaxVoice) but below the clipping threshold166 (Clip) relatively slowly, preferably on the order of about 0.1-0.3dB/sec until the signal has been attenuated approximately 4 dB or thepower level of the signal drops to the comfort zone.

The gain calculator 160 preferably does not adjust the gain factor 152for signals with power levels within the comfort zone (between minVoiceand MaxVoice), or below the maximum noise power threshold 168(MaxNoise). The preferred values of MaxNoise, min Voice, MaxVoice, Clipare related to a noise floor 170 and are preferably in 3 dB increments.The noise floor is preferably empirically derived by calibrating thehost DSP platform with a known load. The noise floor preferablyadjustable and is typically within the range of about, −45 to −52 dBm. AMaxNoise value of two corresponds to a power level 6 dB above the noisefloor 170, whereas a clip level of nine corresponds to 27 dB above noisefloor 170. For signals with power levels below the comfort zone (lessthan minVoice) but above the maximum noise threshold, the gaincalculator 160 preferably increments the gain factor 152 logarithmicallyat a rate of about 0.1-0.3 dB/sec, until the power level of the signalis within the comfort zone or a gain of approximately 10 dB is reached.

In the described exemplary embodiment, the AGC is designed to adaptslowly, although it should adapt fairly quickly if overflow or clippingis detected. From a system point of view, AGC adaptation should be heldfixed if the NLP 72 (see FIG. 6) is activated or the VAD 80 (see FIG. 6)determines that voice is inactive. In addition, the AGC is preferablysensitive to the amplitude of received call progress tones. In thedescribed exemplary embodiment, rapid adaptation may be enabled as afunction of the actual power level of a received call progress tone suchas for example a ring back tone, compared to the power levels set forthin the applicable standards.

3. Voice Activity Detector

In an exemplary embodiment, the VAD, in either the encoder system or thedecoder system, can be configured to operate in multiple modes so as toprovide system tradeoffs between voice quality and bandwidthrequirements. In a first mode, the VAD is always disabled and declaresall digital voice samples as active speech. This mode is applicable ifthe signal processing system is used over a TDM network, a network whichis not congested with traffic, or when used with PCM (ITU RecommendationG.711 (1988)—Pulse Code Modulation (PCM) of Voice Frequencies, thecontents of which is incorporated herein by reference as if set forth infull) in a PCM bypass mode for supporting data or fax modems.

In a second “transparent” mode, the voice quality is indistinguishablefrom the first mode. In transparent mode, the VAD identifies digitalvoice samples with an energy below the threshold of hearing as inactivespeech. The threshold may be adjustable between −90 and −40 dBm with adefault value of −60 dBm. The transparent mode may be used if voicequality is much more important than bandwidth. This may be the case, forexample, if a G.711 voice encoder (or decoder) is used.

In a third “conservative” mode, the VAD identifies low level (butaudible) digital voice samples as inactive, but will be fairlyconservative about discarding the digital voice samples. A lowpercentage of active speech will be clipped at the expense of slightlyhigher transmit bandwidth. In the conservative mode, a skilled listenermay be able to determine that voice activity detection and comfort noisegeneration is being employed. The threshold for the conservative modemay preferably be adjustable between −65 and −35 dBm with a defaultvalue of −60 dBm.

In a fourth “aggressive” mode, bandwidth is at a premium. The VAD isaggressive about discarding digital voice samples which are declaredinactive. This approach will result in speech being occasionallyclipped, but system bandwidth will be vastly improved. The threshold forthe aggressive mode may preferably be adjustable between −60 and −30 dBmwith a default value of −55 dBm.

The transparent mode is typically the default mode when the system isoperating with 16 bit PCM, companded PCM (G.711) or adaptivedifferential PCM (ITU Recommendations G.726 (December 1990)—40, 32, 24,16; kbit/s Using Low-Delay Code Exited Linear Prediction, and G.727(December 1990)—5-, 4-, 3-, and 2-Sample Embedded Adaptive DifferentialPulse Code Modulation). In these instances, the user is most likelyconcerned with high quality voice since a high bit-rate voice encoder(or decoder) has been selected. As such, a high quality VAD should beemployed. The transparent mode should also be used for the VAD operatingin the decoder system since bandwidth is not a concern (the VAD in thedecoder system is used only to update the comfort noise parameters). Theconservative mode could be used with ITU Recommendation G.728 (September1992)—Coding of Speech at 16 kbit/s Using Low-Delay Code Excited LinearPrediction, G.729, and G.723.1. For systems demanding high bandwidthefficiency, the aggressive mode can be employed as the default mode.

The mechanism in which the VAD detects digital voice samples that do notcontain active speech can be implemented in a variety of ways. One suchmechanism entails monitoring the energy level of the digital voicesamples over short periods (where a period length is typically in therange of about 10 to 30 msec). If the energy level exceeds a fixedthreshold, the digital voice samples are declared active, otherwise theyare declared inactive. The transparent mode can be obtained when thethreshold is set to the threshold level of hearing.

Alternatively, the threshold level of the VAD can be adaptive and thebackground noise energy can be tracked. If the energy in the currentperiod is sufficiently larger than the background noise estimate by thecomfort noise estimator, the digital voice samples are declared active,otherwise they are declared inactive. The VAD may also freeze thecomfort noise estimator or extend the range of active periods(hangover). This type of VAD is used in GSM (European Digital CellularTelecommunications System; Half rate Speech Part 6: Voice ActivityDetector (VAD) for Half Rate Speech Traffic Channels (GSM 6.42), thecontents of which is incorporated herein by reference as if set forth infull) and QCELP (W. Gardner, P. Jacobs, and C. Lee, “QCELP: A VariableRate Speech Coder for CDMA Digital Cellular,” in Speech and Audio Codingfor Wireless and Network Applications, B. S. atal, V. Cuperman, and A.Gersho (eds)., the contents of which is incorporated herein by referenceas if set forth in full).

In a VAD utilizing an adaptive threshold level, speech parameters suchas the zero crossing rate, spectral tilt, energy and spectral dynamicsare measured and compared to stored values for noise. If the parametersdiffer significantly from the stored values, it is an indication thatactive speech is present even if the energy level of the digital voicesamples is low.

When the VAD operates in the conservative or transparent mode, measuringthe energy of the digital voice samples can be sufficient for detectinginactive speech. However, the spectral dynamics of the digital voicesamples against a fixed threshold may be useful in discriminatingbetween long voice segments with audio spectra and long term backgroundnoise. In an exemplary embodiment of a VAD employing spectral analysis,the VAD performs auto-correlations using Itakura or Itakura-Saitodistortion to compare long term estimates based on background noise toshort term estimates based on a period of digital voice samples. Inaddition, if supported by the voice encoder, line spectrum pairs (LSPs)can be used to compare long term LSP estimates based on background noiseto short terms estimates based on a period of digital voice samples.Alternatively, FFT methods can be are used when the spectrum isavailable from another software module.

Preferably, hangover should be applied to the end of active periods ofthe digital voice samples with active speech. Hangover bridges shortinactive segments to ensure that quiet trailing, unvoiced sounds (suchas /s/), are classified as active. The amount of hangover can beadjusted according to the mode of operation of the VAD. If a periodfollowing a long active period is clearly inactive (i.e., very lowenergy with a spectrum similar to the measured background noise) thelength of the hangover period can be reduced. Generally, a range ofabout 40 to 300 msec of inactive speech following an active speech burstwill be declared active speech due to hangover.

4. Comfort Noise Generator

According to industry research the average voice conversation includesas much as sixty percent silence or inactive content so thattransmission across the packet based network can be significantlyreduced if non-active speech packets are not transmitted across thepacket based network. In an exemplary embodiment of the presentinvention, a comfort noise generator is used to effectively reproducebackground noise when non-active speech packets are not received. In thedescribed preferred embodiment. comfort noise is generated as a functionsignal characteristics received from a remote source and estimatedsignal characteristics. In the described exemplary embodiment comfortnoise parameters are preferably generated by a comfort noise estimator.The comfort noise parameters may be transmitted from the far end or canbe generated by monitoring the energy level and spectral characteristicsof the far end noise at the end of active speech (i.e., during thehangover period). Although comfort noise generation is described in thecontext of a signal processing system for packet voice exchange, thoseskilled in the art will appreciate that the techniques described forcomfort noise generation are likewise suitable for various applicationsrequiring reconstruction of a signal from signal parameters.Accordingly, the described exemplary embodiment for comfort noisegeneration in a signal processing system for voice applications is byway of example only and not by way of limitation.

A comfort noise generator plays noise. In an exemplary embodiment, acomfort noise generator in accordance with ITU standards G.729 Annex Bor G.723.1 Annex A may be used. These standards specify background noiselevels and spectral content. Referring to FIG. 6, the VAD 80 in theencoder system determines whether the digital voice samples in the mediaqueue 66 contain active speech. If the VAD 80 determines that thedigital voice samples do not contain active speech, then the comfortnoise estimator 81 estimates the energy and spectrum of the backgroundnoise parameters at the near end to update a long running backgroundnoise energy and spectral estimates. These estimates are periodicallyquantized and transmitted in a SID packet by the comfort noise estimator(usually at the end of a talk spurt and periodically during the ensuingsilent segment, or when the background noise parameters changeappreciably). The comfort noise estimator 81 should update the longrunning averages, when necessary, decide when to transmit a SID packet,and quantize and pass the quantized parameters to the packetizationengine 78. SID packets should not be sent while the near end telephonydevice is on-hook, unless they are required to keep the connectionbetween the telephony devices alive. There may be multiple quantizationmethods depending on the protocol chosen.

In many instances the characterization of spectral content or energylevel of the background noise may not be available to the comfort noisegenerator in the decoder system. For example, SID packets may not beused or the contents of the SID packet may not be specified (seeFRF-11). Similarly, the SID packets may only contain an energy estimate,so that estimating some or all of the parameters of the noise in thedecoding system may be necessary. Therefore, the comfort noise generator92 (see FIG. 6) preferably should not be dependent upon SID packets fromthe far end encoder system for proper operation.

In the absence of SID packets, or SID packets containing energy only,the parameters of the background noise at the far end may be estimatedby either of two alternative methods. First, the VAD 98 at the voicedecoder 96 can be executed in series with the comfort noise estimator100 to identify silence periods and to estimate the parameters of thebackground noise during those silence periods. During the identifiedinactive periods, the digital samples from the voice decoder 96 are usedto update the comfort noise parameters of the comfort noise estimator.The far end voice encoder should preferably ensure that a relativelylong hangover period is used in order to ensure that there arenoise-only digital voice samples which the VAD 98 may identify asinactive speech.

Alternatively, in the case of SID packets containing energy levels only,the comfort noise estimate may be updated with the two or three digitalvoice frames which arrived immediately prior to the SID packet. The farend voice encoder should preferably ensure that at least two or threeframes of inactive speech are transmitted before the SID packet istransmitted. This can be realized by extending the hangover period. Thecomfort noise estimator 100 may then estimate the parameters of thebackground noise based upon the spectrum and or energy level of theseframes. In this alternate approach continuous VAD execution is notrequired to identify silence periods, so as to further reduce theaverage bandwidth required for a typical voice channel.

Alternatively, if it is unknown whether or not the far end voice encodersupports (sending) SID packets, the decoder system may start with theassumption that SID packets are not being sent, utilizing a VAD toidentify silence periods, and then only use the comfort noise parameterscontained in the SID packets if and when a SID packet arrives.

A preferred embodiment of the comfort noise generator generates comfortnoise based upon the energy level of the background noise containedwithin the SID packets and spectral information derived from thepreviously decoded inactive speech frames. The described exemplaryembodiment (in the decoding system) includes a comfort noise estimatorfor noise analysis and a comfort noise generator for noise synthesis.Preferably there is an extended hangover period during which the decodedvoice samples is primarily inactive before the VAD identifies the signalas being inactive, (changing from speech to noise). Linear PredictionCoding (LPC) coefficients may be used to model the spectral shape of thenoise during the hangover period just before the SID packet is receivedfrom the VAD. Linear prediction coding models each voice sample as alinear combination of previous samples, that is, as the output of anall-pole IIR filter. Referring to FIG. 10, a noise analyzer 174determines the LPC coefficients.

In the described exemplary embodiment of the comfort noise estimator inthe decoding system, a signal buffer 176 receives and buffers decodedvoice samples. An energy estimator 177 analyzes the energy level of thesamples buffered in the signal buffer 176. The energy estimator 177compares the estimated energy level of the samples stored in the signalbuffer with the energy level provided in the SID packet. Comfort noiseestimating is terminated if the energy level estimated for the samplesstored in the signal buffer and the energy level provided in the SIDpacket differ by more than a predetermined threshold, preferably on theorder of about 6 dB. In addition, the energy estimator 177, analyzes thestability of the energy level of the samples buffered in the signalbuffer. The energy estimator 177 preferably divides the samples storedin the signal buffer into two groups, (preferably approximately equalhalves) and estimates the energy level for each group. Comfort noiseestimation is preferably terminated if the estimated energy levels ofthe two groups differ by more than a predetermined threshold, preferablyon the order of about 6 dB. A shaping filter 178 filters the incomingvoice samples from the energy estimator 177 with a triangular windowingtechnique. Those of skill in the art will appreciate that alternativeshaping filters such as, for example, a Hamming window, may be used toshape the incoming samples.

When a SID packet is received in the decoder system, auto correlationlogic 179 calculates the auto-correlation coefficients of the windowedvoice samples. The signal buffer 176 should preferably be sized to besmaller than the hangover period, to ensure that the auto correlationlogic 179 computes auto correlation coefficients using only voicesamples from the hangover period. In the described exemplary embodiment,the signal buffer is sized to store on the order of about two hundredvoice samples (25 msec assuming a sample rate of 8000 Hz).Autocorrelation, as is known in the art, involves correlating a signalwith itself. A correlation function shows how similar two signals areand how long the signals remain similar when one is shifted with respectto the other. Random noise is defined to be uncorrelated, that is randomnoise is only similar to itself with no shift at all. A shift of onesample results in zero correlation, so that the autocorrelation functionof random noise is a single sharp spike at shift zero. Theautocorrelation coefficients are calculated according to the followingequation:

${r(k)} = {\sum\limits_{n = k}^{m}{{s(n)}{s\left( {n - k} \right)}}}$

where k=0 . . . p and p is the order of the synthesis filter 188 (seeFIG. 11) utilized to synthesize the spectral shape of the backgroundnoise from the LPC filter coefficients.

Filter logic 180 utilizes the auto correlation coefficients to calculatethe LPC filter coefficients 180(a) and prediction gain 180(b) using theLevinson-Durbin Recursion method. Preferrably, the filter logic 180first preferably applies a white noise correction factor to r(0) toincrease the energy level of r(0) by a predetermined amount. Thepreferred white noise correction factor is on the order of about(257/256) which corresponds to a white noise level of approximately 24dB below the average signal power. The white noise correction factoreffectively raises the spectral minima so as to reduce the spectraldynamic range of the auto correlation coefficients to alleviateill-conditioning of the Levinson-Durbin recursion. As is known in theart, the Levinson-Durbin recursion is an algorithm for finding anall-pole IIR filter with a prescribed deterministic autocorrelationsequence. The described exemplary embodiment preferably utilizes a tenthorder (i.e. ten tap) synthesis filter 188. However, a lower order filtermay be used to realize a reduced complexity comfort noise estimator.

The signal buffer 176 should preferably be updated each time the voicedecoder is invoked during periods of active speech. Therefore, whenthere is a transition from speech to noise, the buffer 176 contains thevoice samples from the most recent hangover period. The comfort noiseestimator should preferably ensure that the LPC filter coefficients isdetermined using only samples of background noise. If the LPC filtercoefficients are determined based on the analysis of active speechsamples, the estimated LPC filter coefficients will not give the correctspectrum of the background noise. In the described exemplary embodiment,a hangover period in the range of about 50-250 msec is assumed, andtwelve active frames (assuming 5 msec frames) are accumulated before thefilter logic 180 calculates new LPC coefficients.

In the described exemplary embodiment a comfort noise generator utilizesthe power level of the background noise retrieved from processed SIDpackets and the predicted LPC filter coefficients 180(a) to generatecomfort noise in accordance with the following formula:

${s(n)} = {{e(n)} + {\sum\limits_{i = 1}^{M}{{a(i)}{s\left( {n - i} \right)}}}}$

Where M is the order (i.e. the number of taps) of the synthesis filter188, s(n) is the predicted value of the synthesized noise, a(i) is thei^(th) LPC filter coefficient, s(n−i) are the previous output samples ofthe synthesis filter and e(n) is a Gaussian excitation signal.

A block diagram of the described exemplary embodiment of the comfortnoise generator 182 is shown in FIG. 11. The comfort noise estimatorprocesses SID packets to decode the power level of the current far endbackground noise. The power level of the background noise is forwardedto a power controller 184. In addition a white noise generator 186forwards a gaussian signal to the power controller 184. The powercontroller 184 adjusts the power level of the gaussian signal inaccordance with the power level of the background noise and theprediction gain 180(b). The prediction gain is the difference in powerlevel of the input and output of synthesis filter 188. The synthesisfilter 188 receives voice samples from the power controller 184 and theLPC filter coefficients calculated by the filter logic 180 (see FIG.10). The synthesis filter 188 generates a power adjusted signal whosespectral characteristics approximate the spectral shape of thebackground noise in accordance with the above equation (i.e. sum of theproduct of the LPC filter coefficients and the previous output samplesof the synthesis filter).

5. Voice Encoder/Voice Decoder

The purpose of voice compression algorithms is to represent voice withhighest efficiency (i.e., highest quality of the reconstructed signalusing the least number of bits). Efficient voice compression was madepossible by research starting in the 1930's that demonstrated that voicecould be characterized by a set of slowly varying parameters that couldlater be used to reconstruct an approximately matching voice signal.Characteristics of voice perception allow for lossy compression withoutperceptible loss of quality.

Voice compression begins with an analog-to-digital converter thatsamples the analog voice at an appropriate rate (usually 8,000 samplesper second for telephone bandwidth voice) and then represents theamplitude of each sample as a binary code that is transmitted in aserial fashion. In communications systems, this coding scheme is calledpulse code modulation (PCM).

When using a uniform (linear) quantizer in which there is uniformseparation between amplitude levels. This voice compression algorithm isreferred to as “linear”, or “linear PCM”. Linear PCM is the simplest andmost natural method of quantization. The drawback is that thesignal-to-noise ratio (SNR) varies with the amplitude of the voicesample. This can be substantially avoided by using non-uniformquantization known as companded PCM.

In companded PCM, the voice sample is compressed to logarithmic scalebefore transmission, and expanded upon reception. This conversion tologarithmic scale ensures that low-amplitude voice signals are quantizedwith a minimum loss of fidelity, and the SNR is more uniform across allamplitudes of the voice sample. The process of compressing and expandingthe signal is known as “companding” (COMpressing and exPANDing). Thereexists a worldwide standard for companded PCM defined by the CCITT (theInternational Telegraph and Telephone Consultative Committee).

The CCITT is a Geneva-based division of the InternationalTelecommunications Union (ITU), a New York-based United Nationsorganization. The CCITT is now formally known as the ITU-T, thetelecommunications sector of the ITU, but the term CCITT is still widelyused. Among the tasks of the CCITT is the study of technical andoperating issues and releasing recommendations on them with a view tostandardizing telecommunications on a worldwide basis. A subset of thesestandards is the G-Series Recommendations, which deal with the subjectof transmission systems and media, and digital systems and networks.Since 1972, there have been a number of G-Series Recommendations onspeech coding, the earliest being Recommendation G.711. G.711 has thebest voice quality of the compression algorithms but the highest bitrate requirement.

The ITU-T defined the “first” voice compression algorithm for digitaltelephony in 1972. It is companded PCM defined in Recommendation G.711.This Recommendation constitutes the principal reference as far astransmission systems are concerned. The basic principle of the G.711companded PCM algorithm is to compress voice using 8 bits per sample,the voice being sampled at 8 kHz, keeping the telephony bandwidth of300-3400 Hz. With this combination, each voice channel requires 64kilobits per second.

Note that when the term PCM is used in digital telephony, it usuallyrefers to the companded PCM specified in Recommendation G.711, and notlinear PCM, since most transmission systems transfer data in thecompanded PCM format. Companded PCM is currently the most commondigitization scheme used in telephone networks. Today, nearly everytelephone call in North America is encoded at some point along the wayusing G.711 companded PCM.

ITU Recommendation G.726 specifies a multiple-rate ADPCM compressiontechnique for converting 64 kilobit per second companded PCM channels(specified by Recommendation G.711) to and from a 40, 32, 24, or 16kilobit per second channel. The bit rates of 40, 32, 24, and 16 kilobitsper second correspond to 5, 4, 3, and 2 bits per voice sample.

ADPCM is a combination of two methods: Adaptive Pulse Code Modulation(APCM), and Differential Pulse Code Modulation (DPCM). Adaptive PulseCode Modulation can be used in both uniform and non-uniform quantizersystems. It adjusts the step size of the quantizer as the voice sampleschange, so that variations in amplitude of the voice samples, as well astransitions between voiced and unvoiced segments, can be accommodated.In DPCM systems, the main idea is to quantize the difference betweencontiguous voice samples. The difference is calculated by subtractingthe current voice sample from a signal estimate predicted from previousvoice sample. This involves maintaining an adaptive predictor (which islinear, since it only uses first-order functions of past values). Thevariance of the difference signal results in more efficient quantization(the signal can be compressed coded with fewer bits).

The G.726 algorithm reduces the bit rate required to transmitintelligible voice, allowing for more channels. The bit rates of 40, 32,24, and 16 kilobits per second correspond to compression ratios of1.6:1, 2:1, 2.67:1, and 4:1 with respect to 64 kilobits per secondcompanded PCM. Both G.711 and G.726 are waveform encoders; they can beused to reduce the bit rate require to transfer any waveform, likevoice, and low bit-rate modem signals, while maintaining an acceptablelevel of quality.

There exists another class of voice encoders, which model the excitationof the vocal tract to reconstruct a waveform that appears very similarwhen heard by the human ear, although it may be quite different from theoriginal voice signal. These voice encoders, called vocoders, offergreater voice compression while maintaining good voice quality, at thepenalty of higher computational complexity and increased delay.

For the reduction in bit rate over G.711, one pays for an increase incomputational complexity. Among voice encoders, the G.726 ADPCMalgorithm ranks low to medium on a relative scale of complexity, withcompanded PCM being of the lowest complexity and code-excited linearprediction (CELP) vocoder algorithms being of the highest.

The G.726 ADPCM algorithm is a sample-based encoder like the G.711algorithm, therefore, the algorithmic delay is limited to one sampleinterval. The CELP algorithms operate on blocks of samples (0.625 ms to30 ms for the ITU coder), so the delay they incur is much greater.

The quality of G.726 is best for the two highest bit rates, although itis not as good as that achieved using companded PCM. The quality at 16kilobits per second is quite poor (a noticeable amount of noise isintroduced), and should normally be used only for short periods when itis necessary to conserve network bandwidth (overload situations).

The G.726 interface specifies as input to the G.726 encoder (and outputto the G.726 decoder) an 8-bit companded PCM sample according toRecommendation G.711. So strictly speaking, the G.726 algorithm is atranscoder, taking log-PCM and converting it to ADPCM, and vice-versa.Upon input of a companded PCM sample, the G.726 encoder converts it to a14-bit linear PCM representation for intermediate processing. Similarly,the decoder converts an intermediate 14-bit linear PCM value into an8-bit companded PCM sample before it is output. An extension of theG.726 algorithm was carried out in 1994 to include, as an option, 14-bitlinear PCM input signals and output signals. The specification for sucha linear interface is given in Annex A of Recommendation G.726.

The interface specified by G.726 Annex A bypasses the input and outputcompanded PCM conversions. The effect of removing the companded PCMencoding and decoding is to decrease the coding degradation introducedby the compression and expansion of the linear PCM samples.

The algorithm implemented in the described exemplary embodiment can bethe version specified in G.726 Annex A, commonly referred to as G.726A,or any other voice compression algorithm known in the art. Among thesevoice compression algorithms are those standardized for telephony by theITU-T. Several of these algorithms operate at a sampling rate of 8000Hz. with different bit rates for transmitting the encoded voice. By wayof example, Recommendations G.729 (1996) and G.723.1 (1996) define codeexcited linear prediction (CELP) algorithms that provide even lower bitrates than G.711 and G.726. G.729 operates at 8 kbps and G.723.1operates at either 5.3 kbps or 6.3 kbps.

In an exemplary embodiment, the voice encoder and the voice decodersupport one or more voice compression algorithms, including but notlimited to, 16 bit PCM (non-standard, and only used for diagnosticpurposes); ITU-T standard G.711 at 64 kb/s; G.723.1 at 5.3 kb/s (ACELP)and 6.3 kb/s (MP-MLQ); ITU-T standard G.726 (ADPCM) at 16, 24, 32, and40 kb/s; ITU-T standard G.727 (Embedded ADPCM) at 16, 24, 32, and 40kb/s; ITU-T standard G.728 (LD-CELP) at 16 kb/s; and ITU-T standardG.729 Annex A (CS-ACELP) at 8 kb/s.

The packetization interval for 16 bit PCM, G.711, G.726, G.727 and G.728should be a multiple of 5 msec in accordance with industry standards.The packetization interval is the time duration of the digital voicesamples that are encapsulated into a single voice packet. The voiceencoder (decoder) interval is the time duration in which the voiceencoder (decoder) is enabled. The packetization interval should be aninteger multiple of the voice encoder (decoder) interval (a frame ofdigital voice samples). By way of example, G.729 encodes framescontaining 80 digital voice samples at 8 kHz which is equivalent to avoice encoder (decoder) interval of 10 msec. If two subsequent encodedframes of digital voice sample are collected and transmitted in a singlepacket, the packetization interval in this case would be 20 msec.

G.711, G.726, and G.727 encodes digital voice samples on a sample bysample basis. Hence, the minimum voice encoder (decoder) interval is0.125 msec. This is somewhat of a short voice encoder (decoder)interval, especially if the packetization interval is a multiple of 5msec. Therefore, a single voice packet will contain 40 frames of digitalvoice samples. G.728 encodes frames containing 5 digital voice samples(or 0.625 msec). A packetization interval of 5 msec (40 samples) can besupported by 8 frames of digital voice samples. G.723.1 compressesframes containing 240 digital voice samples. The voice encoder (decoder)interval is 30 msec, and the packetization interval should be a multipleof 30 msec.

Packetization intervals which are not multiples of the voice encoder (ordecoder) interval can be supported by a change to the packetizationengine or the depacketization engine. This may be acceptable for a voiceencoder (or decoder) such as G.711 or 16 bit PCM.

The G.728 standard may be desirable for some applications. G.728 is usedfairly extensively in proprietary voice conferencing situations and itis a good trade-off between bandwidth and quality at a rate of 16 kb/s.Its quality is superior to that of G.729 under many conditions, and ithas a much lower rate than G.726 or G.727. However, G.728 is MIPSintensive.

Differentiation of various voice encoders (or decoders) may come at areduced complexity. By way of example, both G.723.1 and G.729 could bemodified to reduce complexity, enhance performance, or reduce possibleIPR conflicts. Performance may be enhanced by using the voice encoder(or decoder) as an embedded coder. For example, the “core” voice encoder(or decoder) could be G.723.1 operating at 5.3 kb/s with “enhancement”information added to improve the voice quality. The enhancementinformation may be discarded at the source or at any point in thenetwork, with the quality reverting to that of the “core” voice encoder(or decoder). Embedded coders may be readily implemented since they arebased on a given core. Embedded coders are rate scalable, and are wellsuited for packet based networks. If a higher quality 16 kb/s voiceencoder (or decoder) is required, one could use G.723.1 or G.729 Annex Aat the core, with an extension to scale the rate up to 16 kb/s (orwhatever rate was desired).

The configurable parameters for each voice encoder or decoder includethe rate at which it operates (if applicable), which companding schemeto use, the packetization interval, and the core rate if the voiceencoder (or decoder) is an embedded coder. For G.727, the configurationis in terms of bits/sample. For example EADPCM(5,2) (Embedded ADPCM,G.727) has a bit rate of 40 kb/s (5 bits/sample) with the coreinformation having a rate of 16 kb/s (2 bits/sample).

6. Packetization Engine

In an exemplary embodiment, the packetization engine groups voice framesfrom the voice encoder, and with information from the VAD, creates voicepackets in a format appropriate for the packet based network. The twoprimary voice packet formats are generic voice packets and SID packets.The format of each voice packet is a function of the voice encoder used,the selected packetization interval, and the protocol.

Those skilled in the art will readily recognize that the packetizationengine could be implemented in the host. However, this may unnecessarilyburden the host with configuration and protocol details, and therefore,if a complete self contained signal processing system is desired, thenthe packetization engine should be operated in the network VHD.Furthermore, there is significant interaction between the voice encoder,the VAD, and the packetization engine, which further promotes thedesirability of operating the packetization engine in the network VHD

The packetization engine may generate the entire voice packet or justthe voice portion of the voice packet. In particular, a fully packetizedsystem with all the protocol headers may be implemented, oralternatively, only the voice portion of the packet will be delivered tothe host. By way of example, for VoIP, it is reasonable to create thereal-time transport protocol (RTP) encapsulated packet with thepacketization engine, but have the remaining transmission controlprotocol/Internet protocol (TCP/IP) stack residing in the host. In thedescribed exemplary embodiment, the voice packetization functions residein the packetization engine. The voice packet should be formattedaccording to the particular standard, although not all headers or allcomponents of the header need to be constructed.

7. Voice Depacketizing Engine/Voice Queue

In an exemplary embodiment, voice de-packetization and queuing is a realtime task which queues the voice packets with a time stamp indicatingthe arrival time. The voice queue should accurately identify packetarrival time within one msec resolution. Resolution should preferablynot be less than the encoding interval of the far end voice encoder. Thedepacketizing engine should have the capability to process voice packetsthat arrive out of order, and to dynamically switch between voiceencoding methods (i.e. between, for example, G.723.1 and G.711). Voicepackets should be queued such that it is easy to identify the voiceframe to be released, and easy to determine when voice packets have beenlost or discarded en route.

The voice queue may require significant memory to queue the voicepackets. By way of example, if G.711 is used, and the worst case delayvariation is 250 msec, the voice queue should be capable of storing upto 500 msec of voice frames. At a data rate of 64 kb/s this translatesinto 4000 bytes or, or 2K (16 bit) words of storage. Similarly, for 16bit PCM, 500 msec of voice frames require 4K words. Limiting the amountof memory required may limit the worst case delay variation of 16 bitPCM and possibly G.711 This, however, depends on how the voice framesare queued, and whether dynamic memory allocation is used to allocatethe memory for the voice frames. Thus, it is preferable to optimize thememory allocation of the voice queue.

The voice queue transforms the voice packets into frames of digitalvoice samples. If the voice packets are at the fundamental encodinginterval of the voice frames, then the delay jitter problem issimplified. In an exemplary embodiment, a double voice queue is used.The double voice queue includes a secondary queue which time stamps andtemporarily holds the voice packets, and a primary queue which holds thevoice packets, time stamps, and sequence numbers. The voice packets inthe secondary queue are disassembled before transmission to the primaryqueue. The secondary queue stores packets in a format specific to theparticular protocol, whereas the primary queue stores the packets in aformat which is largely independent of the particular protocol.

In practice, it is often the case that sequence numbers are includedwith the voice packets, but not the SID packets, or a sequence number ona SID packet is identical to the sequence number of a previouslyreceived voice packet. Similarly, SID packets may or may not containuseful information. For these reasons, it may be useful to have aseparate queue for received SID packets.

The depacketizing engine is preferably configured to support VoIP, VTOA,VoFR and other proprietary protocols. The voice queue should be memoryefficient, while providing the ability to dynamically switch betweenvoice encoders (at the far end), allow efficient reordering of voicepackets (used for VoIP) and properly identify lost packets.

8. Voice Synchronization

In an exemplary embodiment, the voice synchronizer analyzes the contentsof the voice queue and determines when to release voice frames to thevoice decoder, when to play comfort noise, when to perform frame repeats(to cope with lost voice packets or to extend the depth of the voicequeue), and when to perform frame deletes (in order to decrease the sizeof the voice queue). The voice synchronizer manages the asynchronousarrival of voice packets. For those embodiments which are not memorylimited, a voice queue with sufficient fixed memory to store the largestpossible delay variation is used to process voice packets which arriveasynchronously. Such an embodiment includes sequence numbers to identifythe relative timings of the voice packets. The voice synchronizer shouldensure that the voice frames from the voice queue can be reconstructedinto high quality voice, while minimizing the end-to-end delay. Theseare competing objectives so the voice synchronizer should be configuredto provide system trade-off between voice quality and delay.

Preferably, the voice synchronizer is adaptive rather than fixed basedupon the worst case delay variation. This is especially true in casessuch as VoIP where the worst case delay variation can be on the order ofa few seconds. By way of example, consider a VoIP system with a fixedvoice synchronizer based on a worst case delay variation of 300 msec. Ifthe actual delay variation is 280 msec, the signal processing systemoperates as expected. However, if the actual delay variation is 20 msec,then the end -to-end delay is at least 280 msec greater than required.In this case the voice quality should be acceptable, but the delay wouldbe undesirable. On the other hand, if the delay variation is 330 msecthen an underflow condition could exist degrading the voice quality ofthe signal processing system.

The voice synchronizer performs four primary tasks. First, the voicesynchronizer determines when to release the first voice frame of a talkspurt from the far end. Subsequent to the release of the first voiceframe, the remaining voice frames are released in an isochronous manner.In an exemplary embodiment, the first voice frame is held for a periodof time that is equal or less than the estimated worst case jitter.

Second, the voice synchronizer estimates how long the first voice frameof the talk spurt should be held. If the voice synchronizerunderestimates the required “target holding time,” jitter bufferunderflow will likely result. However, jitter buffer underflow couldalso occur at the end of a talk spurt, or during a short silenceinterval. Therefore, SID packets and sequence numbers could be used toidentify what caused the jitter buffer underflow, and whether the targetholding time should be increased. If the voice synchronizeroverestimates the required “target holding time,” all voice frames willbe held too long causing jitter buffer overflow. In response to jitterbuffer overflow, the target holding time should be decreased. In thedescribed exemplary embodiment, the voice synchronizer increases thetarget holding time rapidly for jitter buffer underflow due to excessivejitter, but decreases the target holding time slowly when holding timesare excessive. This approach allows rapid adjustments for voice qualityproblems while being more forgiving for excess delays of voice packets.

Thirdly, the voice synchronizer provides a methodology by which framerepeats and frame deletes are performed within the voice decoder.Estimated jitter is only utilized to determine when to release the firstframe of a talk spurt. Therefore, changes in the delay variation duringthe transmission of a long talk spurt must be independently monitored.On buffer underflow (an indication that delay variation is increasing),the voice synchronizer instructs the lost frame recovery engine to issuevoice frames repeats. In particular, the frame repeat command instructsthe lost frame recovery engine to utilize the parameters from theprevious voice frame to estimate the parameters of the current voiceframe. Thus, if frames 1, 2 and 3 are normally transmitted and frame 3arrives late, frame repeat is issued after frame number 2, and if framenumber 3 arrives during this period, it is then transmitted. Thesequence would be frames 1, 2, a frame repeat of frame 2 and then frame3. Performing frame repeats causes the delay to increase, whichincreasing the size of the jitter buffer to cope with increasing delaycharacteristics during long talk spurts. Frame repeats are also issuedto replace voice frames that are lost en route.

Conversely, if the holding time is too large due to decreasing delayvariation, the speed at which voice frames are released should beincreased. Typically, the target holding time can be adjusted, whichautomatically compresses the following silent interval. However, duringa long talk spurt, it may be necessary to decrease the holding time morerapidly to minimize the excessive end to end delay. This can beaccomplished by passing two voice frames to the voice decoder in onedecoding interval but only one of the voice frames is transferred to themedia queue.

The voice synchronizer must also function under conditions of severebuffer overflow, where the physical memory of the signal processingsystem is insufficient due to excessive delay variation. When subjectedto severe buffer overflow, the voice synchronizer could simply discardvoice frames.

The voice synchronizer should operate with or without sequence numbers,time stamps, and SID packets. The voice synchronizer should also operatewith voice packets arriving out of order and lost voice packets. Inaddition, the voice synchronizer preferably provides a variety ofconfiguration parameters which can be specified by the host for optimumperformance, including minimum and maximum target holding time. Withthese two parameters, it is possible to use a fully adaptive jitterbuffer by setting the minimum target holding time to zero msec and themaximum target holding time to 500 msec (or the limit imposed due tomemory constraints). Although the preferred voice synchronizer is fullyadaptive and able to adapt to varying network conditions, those skilledin the art will appreciate that the voice synchronizer can also bemaintained at a fixed holding time by setting the minimum and maximumholding times to be equal.

9. Lost Packet Recovery/Frame Deletion

In applications where voice is transmitted through a packet basednetwork there are instances where not all of the packets reach theintended destination. The voice packets may either arrive too late to besequenced properly or may be lost entirely. These losses may be causedby network congestion, delays in processing or a shortage of processingcycles. The packet loss can make the voice difficult to understand orannoying to listen to.

Packet recovery refers to methods used to hide the distortions caused bythe loss of voice packets. In the described exemplary embodiment, a lostpacket recovery engine is implemented whereby missing voice is filledwith synthesized voice using the linear predictive coding model ofspeech. The voice is modelled using the pitch and spectral informationfrom digital voice samples received prior to the lost packets.

The lost packet recovery engine, in accordance with an exemplaryembodiment, can be completely contained in the decoder system. Thealgorithm uses previous digital voice samples or a parametricrepresentation thereof, to estimate the contents of lost packets whenthey occur.

FIG. 12 shows a block diagram of the voice decoder and the lost packetrecovery engine. The lost packet recovery engine includes a voiceanalyzer 192, a voice synthesizer 194 and a selector 196. During periodsof no packet loss, the voice analyzer 192 buffers digital voice samplesfrom the voice decoder 96.

When a packet loss occurs, the voice analyzer 192 generates voiceparameters from the buffered digital voice samples. The voice parametersare used by the voice synthesizer 194 to synthesize voice until thevoice decoder 96 receives a voice packet, or a timeout period haselapsed. During voice syntheses, a “packet lost” signal is applied tothe selector to output the synthesized voice as digital voice samples tothe media queue (not shown).

A flowchart of the lost recovery engine algorithm is shown in FIG. 13A.The algorithm is repeated every frame, whether or not there has been alost packet. Every time the algorithm is performed, a frame of digitalvoice samples are output. For purposes of explanation, assume a framelength of 5 ms. In this case, forty samples (5 ms of samples for asampling rate of 8000 Hz) and a flag specifying whether or not there isvoice is buffered in the voice analyzer. The output of the lost recoveryengine is also forty digital voice samples.

First, a check is made to see if there has been a packet loss 191. Ifso, then a check is made to see if this is the first lost packet in aseries of voice packets 193. If it is the first lost packet, then thevoice is analysed by calculating the LPC parameters, the pitch, and thevoicing decision 195 of the buffered digital samples. If the digitalsamples are voiced 197, then a residual signal is calculated 199 fromthe buffered digital voice samples and an excitation signal is createdfrom the residual signal 201. The gain factor for the excitation is setto one. If the speech is unvoiced 197, then the excitation gain factoris determined from a prediction error power calculated during aLevinson-Durbin recursion process 207. Using the parameters determinedfrom the voice analysis, one frame of voice is synthesized 201. Finally,the excitation gain factor is attenuated 203, and the synthesizeddigital voice samples are output 205.

If this is not the first lost packet 193, then a check is made on howmany packets have been lost. If the number of lost packets exceeds athreshold 209, then a silence signal is generated and output 211.Otherwise, a frame of digital voice samples are synthesized 201, theexcitation gain factor is attenuated 203, and the synthesized digitalvoice samples are output 205.

If there are decoded digital voice samples 191, then a check isperformed to see if there was a lost packet the last time the algorithmwas executed 213. If so, then one-half of a frame of digital voicesamples are synthesized, and overlap-added with the first one-half ofthe frame of decoded digital voice samples 215. Then, in all cases, thedigital voice samples are buffered in the voice analyser and a frame ofdigital voice samples is output 217.

a. Calculation of LPC Parameters

There are two main steps in finding the LPC parameters. First theautocorrelation function r(i) is determined up to r(M) where M is theprediction order. Then the Levinson-Durbin recursion formula is appliedto the autocorrelation function to get the LPC parameters.

There are several steps involved in calculating the autocorrelationfunction. The calculations are performed on the most recent buffereddigital voice samples. First, a Hamming window is applied to thebuffered samples. Then r(0) is calculated and converted to afloating-point format. Next, r(1) to r(M) are calculated and convertedto floating-point. Finally, a conditioning factor is applied to r(0) inorder to prevent ill conditioning of the R matrix for a matrixinversion.

The calculation of the autocorrelation function is preferablycomputationally efficient and makes the best use of fixed pointarithmetic. The following equation is used as an estimate of theautocorrelation function from r(0) to r(M):

${r(i)} = {\sum\limits_{n = 0}^{N - i - 1}{{s\lbrack n\rbrack} \cdot {s\left\lbrack {n - i} \right\rbrack}}}$

where s[n] is the voice signal and N is the length of the voice window.

The value of r(0) is scaled such that it is represented by a mantissaand an exponent. The calculations are performed using 16 bitmultiplications and the summed results are stored in a 40-bit register.The mantissa is found by shifting the result left or right such that themost significant bit is in bit 30 of the 40-bit register (where theleast significant bit is bit 0) and then keeping bits 16 to 31. Theexponent is the number of left shifts required for normalization of themantissa. The exponent may be negative if a large amplitude signal ispresent.

The values calculated for r(1) to r(M) are scaled to use the sameexponent as is used for r(0), with the assumption that all values of theautocorrelation function are less than or equal to r(0). Thisrepresentation in which a series of values are represented with the sameexponent is called block floating-point because the whole block of datais represented using the same exponent.

A conditioning factor of 1025/1024 is applied to r(0) in order toprevent ill conditioning of the R matrix. This factor increases thevalue of r(0) slightly, which has the effect of making r(0) larger thanany other value of r(i). It prevents two rows of the R matrix fromhaving equal values or nearly equal values, which would cause illconditioning of the matrix. When the matrix is ill conditioned, it isdifficult to control the numerical precision of results during theLevinson-Durbin recursion.

Once the autocorrelation values have been calculated, theLevinson-Durbin recursion formula is applied. In the described exemplaryembodiment a sixth to tenth order predictor is preferably used.

Because of truncation effects caused by the use of fixed pointcalculations, errors can occur in the calculations when the R matrix isill conditioned. Although the conditioning factor applied to r(0)eliminates this problem for most cases, there is a numerical stabilitycheck implemented in the recursion algorithm. If the magnitude of thereflection coefficient gets greater than or equal to one, then therecursion is terminated, the LPC parameters are set to zero, and theprediction error power is set to r(0).

b. Pitch Period and Voicing Calculation.

The voicing determination and pitch period calculation are performedusing the zero crossing count and autocorrelation calculations. The twooperations are combined such that the pitch period is not calculated ifthe zero crossing count is high since the digital voice samples areclassified as unvoiced. FIG. 13B shows a flowchart of the operationsperformed.

First the zero crossing count is calculated for a series of digitalvoice samples 219. The zero crossing count is initialized to zero. Thezero crossings are found at a particular point by multiplying thecurrent digital voice sample by the previous digital voice sample andconsidering the sign of the result. If the sign is negative, then therewas a zero crossing and the zero crossing count is incremented. Thisprocess is repeated for a number of digital voice samples, and then thezero crossing count is compared to a pre-determined threshold. If thecount is above the threshold 221, then the digital voice sample isclassified as unvoiced 223. Otherwise, more computations are performed.

Next, if the digital voice samples are not classified as unvoiced, thepitch period is calculated 225. One way to estimate the pitch period ina given segment of speech is to maximize the autocorrelation coefficientover a range of pitch values. This is shown in equation equation below:

$P = {\arg \; {\max_{p}\left( \frac{\sum\limits_{i = 0}^{N - p - 1}{{s\lbrack i\rbrack} \cdot {s\left\lbrack {i + p} \right\rbrack}}}{\sqrt{\sum\limits_{i = 0}^{N - p - 1}{{s\lbrack i\rbrack} \cdot {s\lbrack i\rbrack}}} \cdot \sqrt{\sum\limits_{i = 0}^{N - p - 1}{{s\left\lbrack {i + p} \right\rbrack} \cdot {s\left\lbrack {i + p} \right\rbrack}}}} \right)}}$

An approximation to equation the above equation is used to find thepitch period. First the denominator is approximated by r(0) and thesummation limit in the numerator is made independent of p as follows

$P = {\arg \; {\max_{p}\left( \frac{\sum\limits_{i = 0}^{N - P_{\max} - 1}{{s\lbrack i\rbrack} \cdot {s\left\lbrack {i + p} \right\rbrack}}}{\sum\limits_{i = 0}^{N - P_{\max} - 1}{{s\lbrack i\rbrack} \cdot {s\lbrack i\rbrack}}} \right)}}$

where p is the set of integers greater than or equal to P_(min)(preferably on the order of about 20 samples) and less than or equal toP_(max) (preferably on the order of about 130 samples). Next, thedenominator is removed since it does not depend on p

$P = {\arg \; {\max_{p}\left( {\sum\limits_{i = 0}^{N - P_{\max} - 1}{{s\lbrack i\rbrack} \cdot {s\left\lbrack {i + p} \right\rbrack}}} \right)}}$

Finally, the speech arrays are indexed such that the most recent samplesare emphasized in the estimation of the pitch

$P = {\arg \; {\max_{p}\left( {\sum\limits_{i = 0}^{N - P_{\max} - 1}{{s\left\lbrack {N - 1 - i} \right\rbrack} \cdot {s\left\lbrack {N - 1 - i - p} \right\rbrack}}} \right)}}$

This change improves the performance when the pitch is changing in thevoice segment under analysis.

When the above equation is applied, a further savings in computations ismade by searching only odd values of p. Once the maximum value has beendetermined, a finer search is implemented by searching the two evenvalues of p on either side of the maximum. Although this searchprocedure is non-optimal, it normally works well because theautocorrelation function is quite smooth for voiced segments.

Once the pitch period has been calculated, the voicing decision is madeusing the maximum autocorrelation value 227. If the result is greaterthan 0.38 times r(0) then the digital samples are classified as voiced229. Otherwise it is classified as unvoiced 223.

c. Excitation Signal Calculation.

For voiced samples, the excitation signal for voice synthesis is derivedby applying the following equation to the buffered digital voicesamples:

${e\lbrack n\rbrack} = {{s\lbrack n\rbrack} - {\sum\limits_{i = 1}^{M}{a_{i} \cdot {s\left\lbrack {n - i} \right\rbrack}}}}$

d. Excitation Gain Factor for Unvoiced Speech.

For unvoiced samples, the excitation signal for voice synthesis is awhite Gaussian noise sequence with a variance of one quarter. In orderto synthesize the voice at the correct level, a gain factor is derivedfrom the prediction error power derived during the Levinson-Durbinrecursion algorithm. The prediction error power level gives the powerlevel of the excitation signal that will produce a synthesized voicewith power level r(0). Since a gain level is desired rather than a powerlevel, the square root of the prediction error power level iscalculated. To make up for the fact that the Gaussian noise has a powerof one quarter, the gain is multiplied by a factor of two.

e. Voiced Synthesis.

The voiced synthesis is performed every time there is a lost voicedpacket and also for the first decoded voiced packet after a series oflost packets. FIG. 13C shows the steps performed in the synthesis ofvoice.

First, the excitation signal is generated. If the samples are voiced231, then the excitation is generated from the residual signal 233. Aresidual buffer in the voice analyzer containing the residual signal ismodulo addressed such that the excitation signal is equal to repetitionsof the past residual signal at the pitch period P:

e(n)={e(n−P) for n<P

e(n−2P) for P≦n<2P

e(n−3P) for 2P≦n<3P

If the value of P is less than the number of samples to be synthesized,then the excitation signal is repeated more than once. If P is greaterthan the number of samples to be generated, then less than one pitchperiod is contained in the excitation. In both cases the algorithm keepstrack of the last index into the excitation buffer such that it canbegin addressing at the correct point for the next time voice synthesisis required.

If the samples are unvoiced, then a series of Gaussian noise samples aregenerated 235. Every sample is produced by the addition of twelveuniformly distributed random numbers. Uniformly distributed samples aregenerated using the linear congruential method (Knuth, 9) as shown bythe following equation

X _(n+1)=(aX _(n) +c) mod m

where a is set to 32763, c to zero, and m to 65536. The initial value ofX_(n) is equal to 29. The sequence of random numbers repeats every 16384values, which is the maximum period for the chosen value of m when c isequal to zero. By choosing c not equal to zero the period of repetitioncould be increased to 65536, but 16384 is sufficient for voicesynthesis. The longest segment of voice synthesized by the algorithm istwelve blocks of forty samples, which requires only 5760 uniformlydistributed samples. By setting c to zero, the number of operations tocalculate the Gaussian random sample is reduced by one quarter.

After the excitation has been constructed, the excitation gain factor isapplied to each sample. Finally, the synthesis filter is applied to theexcitation to generate the synthetic voice 237.

f. Overlap-Add Calculation.

The overlap-add process is performed when the first good packet arrivesafter one or more lost packets. The overlap-add reduces thediscontinuity between the end of the synthesized voice and the beginningof the decoded voice. To overlap the two voice signals, additionaldigital voice samples (equal to one-half of a frame) is synthesized andaveraged with the first one-half frame of the decoded voice packet. Thesynthesized voice is multiplied by a down-sloping linear ramp and thedecoded voice is multiplied by an up-sloping linear ramp. Then the twosignals are added together.

10. DTMF

DTMF (dual-tone, multi-frequency) tones are signaling tones carriedwithin the audio band. A dual tone signal is represented by twosinusoidal signals whose frequencies are separated in bandwidth andwhich are uncorrelated to avoid false tone detection. A DTMF signalincludes one of four tones, each having a frequency in a high frequencyband, and one of four tones, each having a frequency in a low frequencyband. The frequencies used for DTMF encoding and detection are definedby the ITU and are widely accepted around the world.

In an exemplary embodiment of the present invention, DTMF detection isperformed by sampling only a portion of each voice frame. This approachresults in improved overall system efficiency by reducing the complexity(MIPS) of the DTMF detection. Although the DTMF is described in thecontext of a signal processing system for packet voice exchange, thoseskilled in the art will appreciate that the techniques described forDMTF are likewise suitable for various applications requiring signaldetection by sampling a portion of the signal. Accordingly, thedescribed exemplary embodiment for DTMF in a signal processing system isby way of example only and not by way of limitation.

There are numerous problems involved with the transmission of DTMF inband over a packet based network. For example, lossy voice compressionmay distort a valid DTMF tone or sequence into an invalid tone orsequence. Also voice packet losses of digital voice samples may corruptDTMF sequences and delay variation (jitter) may corrupt the DTMF timinginformation and lead to lost digits. The severity of the variousproblems depends on the particular voice decoder, the voice decoderrate, the voice packet loss rate, the delay variation, and theparticular implementation of the signal processing system. Forapplications such as VoIP with potentially significant delay variation,high voice packet loss rates, and low digital voice sample rate (ifG.723.1 is used), packet tone exchange is desirable. Packet toneexchange is also desirable for VoFR (FRF-11, class 2). Thus, properdetection and out of band transfer via the packet based network isuseful.

The ITU and Bellcore have promulgated various standards for DTMFdetectors. The described exemplary DTMF detector preferably complieswith ITU-T Standard Q.24 (for DTMF digit reception) and BellcoreGR-506-Core, TR-TSY-000181, TR-TSY-000762 and TR-TSY-000763, thecontents of which are hereby incorporated by reference as though setforth in full herein. These standards involve various criteria, such asfrequency distortion allowance, twist allowance, noise immunity, guardtime, talk-down, talk-off, acceptable signal to noise ratio, and dynamicrange, etc. which are summarized in the table below.

The distortion allowance criteria specifies that a DTMF detector shoulddetect a transmitted signal that has a frequency distortion of less than1.5% and should not detect any DTMF signals that have frequencydistortion of more than 3.5%. The term “twist” refers to the difference,in decibels, between the amplitude of the strongest key pad column toneand the amplitude of the strongest key pad row tone. For example, theBellcore standard requires the twist to be between −8 and +4 dBm. Thenoise immunity criteria requires that if the signal has a signal tonoise ratio (SNR) greater than certain decibels, then the DTMF detectoris required to not miss the signal, i.e., is required to detect thesignal. Different standards have different SNR requirements, whichusually range from 12 to 24 decibels. The guard time check criteriarequires that if a tone has a duration greater than 40 milliseconds, theDTMF detector is required to detect the tone, whereas if the tone has aduration less than 23 milliseconds, the DTMF detector is required to notdetect the tone. Similarly, the DTMF detector is required to acceptinterdigit intervals which are greater than or equal to 40 milliseconds.Alternate embodiments of the present invention readily provide forcompliance with other telecommunication standards such as EIA-464B, andJJ-20.12.

Referring to FIG. 14 the DTMF detector 76 processes the 64 kb/s pulsecode modulated (PCM) signal, i.e., digital voice samples 76(a) bufferedin the media queue (not shown). The input to the DTMF detector 76 shouldpreferably be sampled at a rate that is at least higher thanapproximately 4 kHz or twice the highest frequency of a DTMF tone. Ifthe incoming signal (i.e., digital voice samples) is sampled at a ratethat is greater than 4 kHz (i.e. Nyquist for highest frequency DTMFtone) the signal may immediately be downsampled so as to reduce thecomplexity of subsequent processing. The signal may be downsampled byfiltering and discarding samples.

A block diagram of an exemplary embodiment of the invention is shown inFIG. 14. The described exemplary embodiment includes a system forprocessing the upper frequency band tones and a substantially similarsystem for processing the lower frequency band tones. A filter 210 andsampler 212 may be used to down-sample the incoming signal. In thedescribed exemplary embodiment, the sampling rate is 8 kHz and the frontend filter 210 and sampler 212 do not down-sample the incoming signal.The output of the sampler 212 is filtered by two bandpass filtersH_(h)(z) 214 and G_(h)(z) 216 for the upper frequency band and H_(l)(z)218 and G_(l)(Z) 220 for the lower frequency band) and down-sampled bysamplers 222,224 for the upper frequency band and 226,228 for the lowerfrequency band. The bandpass filters (214,216 and 218,220) for eachfrequency band are designed using a pair of lowpass filters, one filterH(z) which multiplies the down-sampled signal by cos(2πf_(b)nT) and theother filter G(z) which multiplies the down-sampled signal bysin(2πf_(h)nT) (where T=1/f_(s) where f_(s) is the sampling frequencyafter the front end down-sampling by the filter 210 and the sampler 212.

In the described exemplary embodiment, the bandpass filters (214, 216and 218, 220) are executed every eight samples and the outputs (214 a,216 a and 218 a, 220 a) of the bandpass filters (214, 216 and 218, 220)are down-sampled by samplers 222, 224 and 226, 228 at a ratio of eightto one. The combination of down-sampling is selected so as to optimizethe performance of a particular DSP in use and preferably provides asample approximately every msec or a 1 kbs signal. Down-sampled signalsin the upper and lower frequency bands respectively are real signals. Inthe upper frequency band, a multiplier 230 multiplies the output ofsampler 224 by the square root of minus one (i.e. j) 232. A summer 234then adds the output of downsampler 222 with the imaginary signal230(a). Similarly, in the lower frequency band, a multiplier 236multiplies the output of downsampler 228 by the square root of minus one(i.e. j) 238. A summer 240 then adds the output of downsampler 226 withthe imaginary signal 236(a). Combined signals x_(h)(t) 234(a) andx_(l)(t) 240(a) at the output of the summers 234, 240 are complexsignals. It will be appreciated by one of skill in the art that thefunction of the bandpass filters can be accomplished by alternativefinite impulse response filters or structures such as windowing followedby DFT processing.

If a single frequency is present within the bands defined by thebandpass filters, the combined complex signals x_(h)(t) and x_(l)(t)will be constant envelope (complex) signals. Short term power estimator242 and 244 measure the power of x_(h)(t) and x_(l)(t) respectively andcompare the estimated power levels of x_(h)(t) and x_(l)(t) with therequirements promulgated in ITU-T Q.24. In the described exemplaryembodiment, the upper band processing is first executed to determine ifthe power level within the upper band complies with the thresholds setforth in the ITU-T Q.24 recommendations. If the power within the upperband does not comply with the ITU-T recommendations the signal is not aDTMF tone and processing is terminated. If the power within the upperband complies with the ITU-T Q.24 standard, the lower band is processed.A twist estimator 246 compares the power in the upper band and the lowerband to determine if the twist (defined as the ratio of the power in thelower band and the power in the upper band) is within an acceptablerange as defined by the ITU-T recommendations. If the ratio of the powerwithin the upper band and lower band is not within the bounds defined bythe standards, a DTMF tone is not present and processing is terminated.

If the ratio of the power within the upper band and lower band complieswith the thresholds defined by the ITU-T Q.24 and Bellcore GR-506-Core,TR-TSY-000181, TR-TSY-000762 and TR-TSY-000763 standards, the frequencyof the upper band signal x_(h)(t) and the frequency of the lower bandsignal x_(l)(t) are estimated. Because of the duration of the inputsignal (one msec), conventional frequency estimation techniques such ascounting zero crossings may not sufficiently resolve the inputfrequency. Therefore, differential detectors 248 and 250 are used toestimate the frequency of the upper band signal x_(h)(t) and the lowerband signal x_(l)(t) respectively. The differential detectors 248 and250 estimate the phase variation of the input signal over a given timerange. Advantageously, the accuracy of estimation is substantiallyinsensitive to the period over which the estimation is performed. Withrespect to upper band input x_(h)(n), (and assuming x_(h)(n) is asinusoid of frequency f_(l)) the differential detector 248 computes:

y _(h)(n)=x _(h)(n)x _(h)(n−1)*e(−j2πf _(mid))

where f_(mid) is the mean of the frequencies in the upper band or lowerband and superscript* implies complex conjugation. Then,

y _(h)(n)=e(j2πf _(l) n)e(−j2πf _(l)(n−1))e(−j2πf _(mid))=e(j2π(f _(l)−f _(mid)))

which is a constant, independent of n. Arctan functions 252 and 254 eachtakes the complex input and computes the angle of the above complexvalue that uniquely identifies the frequency present in the upper andlower bands. In operation a tan2(sin(2π(f_(l)−f_(mid))),cos(2π(f_(i)−f_(mid)))) returns to within a scaling factor the frequencydifference f_(l)−f_(mid). Those skilled in the art will appreciate thatvarious algorithms, such as a frequency discriminator, could be use toestimate the frequency of the DTMF tone by calculating the phasevariation of the input signal over a given time period.

Having estimated the frequency components of the upper band and lowerband, the DTMF detector analyzes the upper band and lower band signalsto determine whether a DTMF digit is present in the incoming signals andif so which digit. Frequency calculators 256 and 258 compute a mean andvariance of the frequency deviation over the entire window of frequencyestimates to identify valid DTMF tones in the presence of backgroundnoise or speech that resembles a DTMF tone. In the described exemplaryembodiment, if the mean of the frequency estimates over the window iswithin acceptable limits, preferably less than +/−2.8% for the lowbandand +/−2.5% for the highband the variance is computed. If the varianceis less than a predetermined threshold, preferably on the order of about1464 Hz² (i.e. standard deviation of 38.2 Hz) the frequency is declaredvalid. Referring to FIG. 14A, DTMF control logic 259 compares thefrequency identified for the upper and lower bands to the frequencypairs identified in the ITU-T recommendations to identify the digit. TheDTMF control logic 259 forwards a tone detection flag 259(b) to a statemachine 260. The state machine 260 analyzes the time sequence of eventsand compares the tone on and tone off periods for a given tone to theITU-T recommendations to determine whether a valid dual tone is present.In the described exemplary embodiment the total window size ispreferably 5 msec so that a DTMF detection decision is performed every 5msec.

In the context of an exemplary embodiment of the voice mode, the DTMFdetector is operating in the packet tone exchange along with a voiceencoder operating under the packet voice exchange, which allows forsimplification of DTMF detection processing. Most voice encoders operateat a particular frame size (the number of voice samples or time in msecover which voice is compressed). For example, the frame size for ITU-Tstandard G.723.1 is 30 msec. For ITU-T standard G.729 the frame size is10 msec. In addition, many packet voice systems group multiple outputframes from a particular voice encoder into a network cell or packet. Toprevent leakage through the voice path, the described exemplaryembodiment delays DTMF detection until the last frame of speech isprocessed before a full packet is constructed. Therefore, fortransmissions in accordance with the G.723.1 standard and a singleoutput frame placed into a packet, DTMF detection may be invoked every30 msec (synchronous with the end of the frame). Under the G.729standard with two voice encoder frames placed into a single packet, DTMFdetection or decision may be delayed until the end of the second voiceframe within a packet is processed.

In the described exemplary embodiment, the DTMF detector is inherentlystateless, so that detection of DTMF tones within the second 5 msec DTMFblock of a voice encoder frame doesn't depend on DTMF detectorprocessing of the first 5 msec block of that frame. If the delay in DTMFdetection is greater than or equal to twice the DTMF detector blocksize, the processing required for DTMF detection can be furthersimplified. For example, the instructions required to perform DTMFdetection may be reduced by 50% for a voice encoder frame size of 10msec and a DTMF detector frame size of 5 msec. The ITU-T Q.24 standardrequires DTMF tones to have a minimum duration of 23 msec and aninter-digit interval of 40 msec. Therefore, by way of example, a validDTMF tone may be detected within a given 10 msec frame by only analyzingthe second 5 msec interval of that frame. Referring to FIG. 14A, in thedescribed exemplary embodiment, the DTMF control logic 259 analyzes DTMFdetector output 76(a) and selectively enables DTMF detection analysis259(a) for a current frame segment, as a function of whether a validdual tone was detected in previous and future frame segments. Forexample, if a DTMF tone was not detected in the previous frame and ifDTMF is not present in the second 5 msec interval of the current frame,then the first 5 msec block need not be processed so that DTMF detectionprocessing is reduced by 50%. Similar savings may be realized if theprevious frame did contain a DTMF (if the DTMF is still present in thesecond 5 msec portion it is most likely it was on in the first 5 msecportion). This method is easily extended to the case of longer delays(30 msec for G.723.1 or 20-40 msec for G.729 and packetization intervalsfrom 2-4 or more). It may be necessary to search more than one 5 msecperiod out of the longer interval, but only a subset is necessary.

DTMF events are preferably reported to the host. This allows the host,for example, to convert the DTMF sequence of keys to a destinationaddress. It will, therefore, allow the host to support call routing viaDTMF.

Depending on the protocol, the packet tone exchange may support mutingof the received digital voice samples, or discarding voice frames whenDTMF is detected. In addition, to avoid DTMF leakage into the voicepath, the voice packets may be queued (but not released) in the encodersystem when DTMF is pre-detected. DTMF is pre-detected through acombination of DTMF decisions and state machine processing. The DTMFdetector will make a decision (i.e. is there DTMF present) every fivemsec. A state machine 260 analyzes the history of a given DTMF tone todetermine the current duration of a given tone so as to estimate howlong the tone will likely continue. If the detection was false(invalid), the voice packets are ultimately released, otherwise they arediscarded. This will manifest itself as occasional jitter when DTMF isfalsely pre-detected. It will be appreciated by one of skill in the artthat tone packetization can alternatively be accomplished throughcompliance with various industry standards such as for example, theFrame Relay Forum (FRF-11) standard, the voice over atm standard ITU-TI.363.2, and IETF-draft-avt-tone-04, RTP Payload for DTMF Digits forTelephony Tones and Telephony Signals, the contents of which are herebyincorporated by reference as though set forth in full.

Software to route calls via DTMF can be resident on the host or withinthe signal processing system. Essentially, the packet tone exchangetraps DTMF tones and reports them to the host or a higher layer. In anexemplary embodiment, the packet tone exchange will generate dial tonewhen an off-hook condition is detected. Once a DTMF digit is detected,the dial tone is terminated. The packet tone exchange may also have toplay ringing tone back to the near end user (when the far end phone isbeing rung), and a busy tone if the far end phone is unavailable. Othertones may also need to be supported to indicate all circuits are busy,or an invalid sequence of DTMF digits were entered.

11. Call Progress Tone Detection

Telephone systems provide users with feedback about what they are doingin order to simplify operation and reduce calling errors. Thisinformation can be in the form of lights, displays, or ringing, but ismost often audible tones heard on the phone line. These tones aregenerally referred to as call progress tones, as they indicate what ishappening to dialed phone calls. Conditions like busy line, ringingcalled party, bad number, and others each have distinctive tonefrequencies and cadences assigned them for which some standards havebeen established. A call progress tone signal includes one of fourtones. The frequencies used for call progress tone encoding anddetection, namely 350, 440, 480, and 620 Hz, are defined by theinternational telecommunication union and are widely accepted around theworld. The relatively narrow frequency separation between tones, 40 Hzin one instance complicates the detection of individual tones. Inaddition, the duration or cadence of a given tone is used to identifyalternate conditions.

An exemplary embodiment of the call progress tone detector analyzes thespectral (frequency) characteristics of an incoming telephony voice-bandsignal and generates a tone detection flag as a function of the spectralanalysis. The temporal (time) characteristics of the tone detectionflags are then analyzed to detect call progress tone signals. The callprogress tone detector then forwards the call progress tone signal tothe packetization engine to be packetized and transmitted across thepacket based network. Although the call progress tone detector isdescribed in the context of a signal processing system for packet voiceexchange, those skilled in the art will appreciate that the techniquesdescribed for call progress tone detection are likewise suitable forvarious applications requiring signal detection by analyzing spectral ortemporal characteristics of the signal. Accordingly, the describedexemplary embodiment for precision tone detection in a signal processingsystem is by way of example only and not by way of limitation.

The described exemplary embodiment preferably includes a call progresstone detector that operates in accordance with industry standards forthe power level (Bellcore SR3004-CPE Testing Guidelines; Type IIITesting) and cadence (Bellcore GR506-Core and Bellcore LSSGR SignalingFor Analog Interface, Call Purpose Signals) of a call progress tone. Thecall progress tone detector interfaces with the media queue to detectincoming call progress tone signals such as dial tone, re-order tone,audible ringing and line busy or hook status. The problem of callprogress tone signaling and detection is a common telephony problem. Inthe context of packet voice systems in accordance with an exemplaryembodiment of the present invention, telephony devices are coupled to asignal processing system which, for the purposes of explanation, isoperating in a network gateway to support the exchange of voice betweena traditional circuit switched network and a packet based network. Inaddition, the signal processing system operating on network gatewaysalso supports the exchange of voice between the packet based network anda number of telephony devices.

Referring to FIG. 15 the call progress tone detector 264 continuouslymonitors the media queue 66 of the voice encoder system. Typically thecall progress tone detector 264 is invoked every ten msec. Thus, for anincoming signal sampled at a rate of 8 kHz, the preferred call progresstone detector operates on blocks of eighty samples. The call progresstone detector 264 includes a signal processor 266 which analyzes thespectral characteristics of the samples buffered in the media queue 66.The signal processor 266 performs anti-aliasing, decimation, bandpassfiltering, and frequency calculations to determine if a tone at a givenfrequency is present. A cadence processor 268 analyzes the temporalcharacteristics of the processed tones by computing the on and offperiods of the incoming signal. If the cadence processor 268 detects acall progress tone for an acceptable on and off period in accordancewith the Bellcore GR506-Core standard, a “Tone Detection Event” will begenerated.

A block diagram for an exemplary embodiment of the signal processor 266is shown in FIG. 16. An anti-aliasing low pass filter 270, with a cutofffrequency of preferably about 666 Hz, filters the samples buffered inthe media queue so as to remove frequency components above the highestcall progress tone frequency, i.e. 660 Hz. A down sampler 272 is coupledto the output of the low pass filter 270. Assuming an 8 kHz inputsignal, the down sampler 272 preferably decimates the low pass filteredsignal at a ratio of six:one (which avoids aliasing due to undersampling). The output 272(a) of down sampler 272 is filtered by eightbandpass filters (274, 276, 278, 280, 282, 284, 286 and 288), (i.e. twofilters for each call progress tone frequency). The decimationeffectively increases the separation between tones, so as to relax theroll-off requirements (i.e. reduce the number of filter coefficients) ofthe bandpass filters 274-288 which simplifies the identification ofindividual tones. In the described exemplary embodiment, the bandpassfilters for each call progress tone 274-288 are designed using a pair oflowpass filters, one filter which multiplies the down sampled signal bycos(2πf_(h)nT) and the other filter which multiplies the down sampledsignal by sin(2πf_(h)nT) (where T=1/f_(s) where f_(s) is the samplingfrequency after the decimation by the down sampler 272. The outputs ofthe band pass filters are real signals. Multipliers (290, 292, 294 and296) multiply the outputs of filters (276, 280, 284 and 288)respectively by the square root of minus one (i.e. j) 298 to generate animaginary component. Summers (300, 302, 304 and 306) then add theoutputs of filters (274, 278, 282 and 286) with the imaginary components(290 a, 292 a, 294 a and 296 a) respectively. The combined signals arecomplex signals. It will be appreciated by one of skill in the art thatthe function of the bandpass filters (274-288) can be accomplished byalternative finite impulse response filters or structures such aswindowing followed by DFT processing.

Power estimators (308, 310, 312 and 314) estimate the short term averagepower of the combined complex signals (300 a, 302 a, 304 a and 306 a)for comparison to power thresholds determined in accordance with therecommended standard (Bellcore SR3004-CPE Testing Guidelines For TypeIII Testing). The power estimators 308-312 forward an indication topower state machines (316, 318, 320 and 322) respectively which monitorthe estimated power levels within each of the call progress tonefrequency bands. Referring to FIG. 17, the power state machine is athree state device, including a disarm state 324, an arm state 326, anda power on state 328. As is known in the art, the state of a power statemachine depends on the previous state and the new input. For example, ifan incoming signal is initially silent, the power estimator 308 wouldforward an indication to the power state machine 316 that the powerlevel is less than the predetermined threshold. The power state machinewould be off, and disarmed. If the power estimator 308 next detects anincoming signal whose power level is greater than the predeterminedthreshold, the power estimator forwards an indication to the power statemachine 316 indicating that the power level is greater than thepredetermined threshold for the given incoming signal. The power statemachine 316 switches to the off but armed state. If the next input isagain above the predetermined threshold, the power estimator 308forwards an indication to the power state machine 316 indicating thatthe power level is greater than the predetermined threshold for thegiven incoming signal. The power state machine 316 now toggles to the onand armed state. The power state machine 316 substantially reduces oreliminates false detections due to glitches, white noise or other signalanomalies.

Turning back to FIG. 16, when the power state machine is set to the onstate, frequency calculators (330, 332, 334 and 336) estimate thefrequency of the combined complex signals. The frequency calculators(330-336), utilize a differential detection algorithm to estimate thefrequency within each of the four call progress tone bands. Thefrequency calculators (330-336) estimate the phase variation of theinput signal over a given time range. Advantageously, the accuracy ofthe estimation is substantially insensitive to the period over which theestimation is performed. Assuming a sinusoidal input x(n) of frequencyf_(i) the frequency calculator computes:

y(n)=x(n)x(n-1)*e(−j2πf _(mid))

where f_(mid) is the mean of the frequencies within the given callprogress tone group and superscript* implies complex conjugation. Then,

$\begin{matrix}{{y(n)} = {{\left( {{j2\pi}\; f_{1}n} \right)}{\left( {{- {j2\pi}}\; {f_{i}\left( {n - 1} \right)}} \right)}{\left( {{- {j2\pi}}\; f_{mid}} \right)}}} \\{= {\left( {{j2\pi}\left( {f_{1} - f_{mid}} \right)} \right)}}\end{matrix}$

which is a constant, independent of n. The frequency calculators(330-336) then invoke an arctan function that takes the complex signaland computes the angle of the above complex value that identifies thefrequency present within the given call progress tone band. In operationa tan2(sin(2π(f_(i)−f_(mid))), cos(2π(f_(i)−f_(mid)))) returns to withina scaling factor the frequency difference f_(i)−f_(mid). Those skilledin the art will appreciate that various algorithms, such as a frequencydiscriminator, could be use to estimate the frequency of the callprogress tone by calculating the phase variation of the input signalover a given time period.

The frequency calculators (330-336) compute the mean of the frequencydeviation over the entire 10 msec window of frequency estimates toidentify valid call progress tones in the presence of background noiseor speech that resembles a call progress tone. If the mean of thefrequency estimates over the window is within acceptable limits assummarized by the table below, a tone on flag is forwarded to thecadence processor. The frequency calculators (330-336) are preferablyonly invoked if the power state machine is in the on state therebyreducing the processor loading (i.e. fewer MIPS) when a call progresstone signal is not present.

Tone Frequency One/Mean Frequency Two/Mean Dial Tone 350 Hz/2 Hz 440Hz/3 Hz Busy 480 Hz/7 Hz 620 Hz/9 Hz Re-order 480 Hz/7 Hz 620 Hz/9 HzAudible Ringing 440 Hz/7 Hz 480 Hz/7 Hz

Referring to FIG. 18A, the signal processor 266 forwards a tone on/toneoff indication to the cadence processor 268 which considers the timesequence of events to determine whether a call progress tone is present.Referring to FIG. 18, in the described exemplary embodiment, the cadenceprocessor 268 preferably comprises a four state, cadence state machine340, including a cadence tone off state 342, a cadence tone on state344, a cadence tone arm state 346 and an idle state 348 (see FIG. 18).The state of the cadence state machine 340 depends on the previous stateand the new input. For example, if an incoming signal is initiallysilent, the signal processor would forward a tone off indication to thecadence state machine 340. The cadence state machine 340 would be set toa cadence tone off and disarmed state. If the signal processor nextdetects a valid tone, the signal processor forwards a tone on indicationto the cadence state machine 340. The cadence state machine 340 switchesto a cadence off but armed state. Referring to FIG. 18A, the cadencestate machine 340 preferably invokes a counter 350 that monitors theduration of the tone indication. If the next input is again a valid callprogress tone, the signal processor forwards a tone on indication to thecadence state machine 340. The cadence state machine 340 now toggles tothe cadence tone on and cadence tone armed state. The cadence statemachine 340 would remain in the cadence tone on state until receivingtwo consecutive tone off indications from the signal processor at whichtime the cadence state machine 340 sends a tone off indication to thecounter 350. The counter 350, resets and forwards the duration of the ontone to cadence logic 352. The cadence processor 268 similarly estimatesthe duration of the off tone, which the cadence logic 352 utilizes todetermine whether a particular tone is present by comparing the durationof the on tone, off tone signal pair at a given tone frequency to thetone plan recommended in industry standard as summarized in the tablebelow.

Duration of Tone Duration of Tone Tone On/Tolerance Off/Tolerance DialTone Continuous On No Off Tone Busy  500 msec/(+/−50 msec)  500msec/(+/−50 msec) Re-order  250 msec/(+/−25 msec)  200 msec/(+/−25 msec)Audible 1000 msec/(+/−200 msec) 3000 msec/(+/−2000 msec) Ringing Audible2000 msec/(+/−200 msec) 4000 msec/(+/−2000 msec) Ringing (Tone 2)

12. Resource Manager

In the described exemplary embodiment utilizing a multi-layer softwarearchitecture operating on a DSP platform, the DSP server includesnetworks VHDs (see FIG. 2). Each network VHD can be a completeself-contained software module for processing a single channel with anumber of different telephony devices. Multiple channel capability canbe achieved by adding network VHDs to the DSP server. The resourcemanager dynamically controls the creation and deletion of VHDs andservices.

In the case of multi-channel communications using a number of networkVHDs, the services invoked by the network VHDs and the associated PXDsare preferably optimized to minimize system resource requirements interms of memory and/or computational complexity. This can beaccomplished with the resource manager which reduces the complexity ofcertain algorithms in the network VHDs based on predetermined criteria.Although the resource management processor is described in the contextof a signal processing system for packet voice exchange, those skilledin the art will appreciate that the techniques described for resourcemanagement processing are likewise suitable for various applicationsrequiring processor complexity reductions. Accordingly, the describedexemplary embodiment for resource management processing in a signalprocessing system is by way of example only and not by way oflimitation.

In one embodiment, the resource manager can be implemented to reducecomplexity when the worst case system loading exceeds the peak systemresources. The worst case system loading is simply the sum of the worstcase (peak) loading of each service invoked by the network VHD and itsassociated PXDs. However, the statistical nature of the processorresources required to process voice band telephony signals is such thatit is extremely unlikely that the worst case processor loading for eachPXD and/or service will occur simultaneously. Thus, a more robust (loweroverall power consumption and higher densities, i.e. more channels perDSP) signal processing system may be realized if the average complexityof the various voice mode PXDs and associated services is minimized.Therefore, in the described exemplary embodiment, average systemcomplexity is reduced and system resources may be over subscribed (peakloading exceeds peak system resources) in the short term whereincomplexity reductions are invoked to reduce the peak loading placed onthe system.

The described exemplary resource manager should preferably manage theinternal and external program and data memory of the DSP. Thetransmission/signal processing of voice is inherently dynamic, so thatthe system resources required for various stages of a conversation aretime varying. The resource manager should monitor DSP resourceutilization and dynamically allocate resources to numerous VHDs and PXDsto achieve a memory and computationally (reduced MIPS) efficient system.For example, when the near end talker is actively speaking, the voiceencoder consumes significant resources, but the far end is probablysilent so that the echo canceller is probably not adapting and may notbe executing the transversal filter. When the far end is active, thenear end is most likely inactive, which implies the echo canceller isboth canceling far end echo and adapting. However, when the far end isactive the near end is probably inactive, which implies that the VAD isprobably detecting silence and the voice encoder consumes minimal systemresources. Thus, it is unlikely that the voice encoder and echocanceller resource utilization peak simultaneously. Furthermore, ifprocessor resources are taxed, echo canceller adaptation may be disabledif the echo canceller is adequately adapted or interleaved (adaptationenabled on alternating echo canceller blocks) to reduce thecomputational burden placed on the processor.

Referring to FIG. 19, in the described exemplary embodiment, theresource manager 351 manages the resources of two network VHDs 62′, 62″and their associated PXDs 60′, 60″. Initially, the average complexity ofthe services running in each VHD and its associated PXD is reported tothe resource manager. The resource manager 351 sums the reportedcomplexities to determine whether the sum exceeds the system resources.If the sum of the average complexities reported to the resource manager351 are within the capability of the system resources, no complexityreductions are invoked by the resource manager 351. Conversely, if thesum of the average complexities of the services running in each VHD andits associated PXD overload the system resources, then the resourcemanager can invoke a number of complexity reduction methodologies. Forexample, the echo cancellers 70′, 70″ can be forced into the bypass mode(see FIG. 7) and/or the echo canceller adaption can be reduced ordisabled. In addition (or in the alternative), complexity reductions inthe voice encoders 82′, 82″ and voice decoders 96′, 96″ can be invoked.

The described exemplary embodiment may reduce the complexity of certainvoice mode services and associated PXDs so as to reduce thecomputational/memory requirements placed upon the system. Variousmodifications to the voice encoders may be included to reduce the loadplaced upon the system resources. For example, the complexity of aG.723.1 voice encoder may be reduced by disabling the post filter inaccordance with the ITU-T G.723.1 standard which is incorporated hereinby reference as if set forth in full. Also the voicing decision may bemodified so as to be based on the open loop normalized pitch correlationcomputed at the open loop pitch lag L determined by the standard voiceencoding algorithm. This entails a modification to the ITU-T G.723.1 Clanguage routine Estim_Pitch( ). If d(n) is the input to the pitchestimation function, the normalized open loop pitch correlation at theopen loop pitch lag L is:

${X(L)} = \frac{\left( {\sum\limits_{n = 0}^{N - 1}\left( {{d(n)}\left( {{dn} - L} \right)} \right)^{2}} \right.}{\left( {\sum\limits_{n = 0}^{N - 1}{d(n)}^{2}} \right)\left( {\sum\limits_{n = 0}^{N - 1}{d\left( {n - L} \right)}^{2}} \right)}$

where N is equal to a duration of 2 subframes (or 120 samples).

Also, the ability to bypass the adaptive codebook based on a thresholdcomputed from a combination of the open loop normalized pitchcorrelation and speech/residual energy may be included. In the standardencoder, the search through the adaptive codebook gain codebook beginsat index zero and may be terminated before the entire codebook issearched (less than the total size of the adaptive codebook gaincodebook which is either 85 or 170 entries) depending on theaccumulation of potential error. A preferred complexity reductiontruncates the adaptive codebook gain search procedure if the open loopnormalized pitch correlation and speech/residual energy meets a certainby searching entries from:

the upper bound (computed in the standard coder) less half the adaptivecodebook size (or index zero, whichever is greater) for voiced speech;and

from index zero up to half the size of the adaptive code gain codebook(85/2 or 170/2).

The adaptive codebook may also be completely bypassed under someconditions by setting the adaptive codebook gain index to zero, whichselects an all zero adaptive codebook gain setting.

The fixed excitation in the standard encoder may have a periodiccomponent. In the standard encoder, if the open loop pitch lag is lessthan the subframe length minus two, then a excitation search function(the function call Find_Best( ) in the ITU-T G.723.1 C languagesimulation) is invoked twice. To reduce system complexity, the fixedexcitation search procedure may be modified (at 6.3 kb/s) such that thefixed excitation search function is invoked once per invocation of thefixed excitation search procedure (routine Find_Fcbk( )). If the openloop pitch lag is less than the subframe length minus two then aperiodic repetition is forced, otherwise there is no periodic repetition(as per the standard encoder for that range of open loop pitch lags). Inthe described complexity reduction modification, the decision on whichmanner to invoke it is based on the open loop pitch lag and the voicingstrength.

Similarly, the fixed excitation search procedure can be modified (at 5.3kb/s) such that a higher threshold is chosen for voice decisions. In thestandard encoder, the voicing decision is considered to be voiced of theopen loop normalized pitch correlation is greater than 0.5 (variablenamed “threshold” in the ITU-T G.723.1) is set to 0.5. In a modificationto reduce the complexity of this function, the threshold may be set to0.75. This greatly reduces the complexity of the excitation searchprocedure while avoiding substantial impairment to the voice quality.

Similar modifications may be made to reduce the complexity of a G.729Annex A voice encoder. For example, the complexity of a G.729 Annex Avoice encoder may be reduced by disabling the post filter in accordancewith the G.729 Annex A standard which is incorporated herein byreference as if set out in full. Also, the complexity of a G.729 Annex Avoice encoder may be further reduced by including the ability to bypassthe adaptive codebook or reduce the complexity of the adaptive codebooksearch significantly. In the standard voice encoder, the adaptivecodebook searches over a range of lags based on the open loop pitch lag.The adaptive codebook bypass simply chooses the minimum lag. Thecomplexity of the adaptive codebook search may be reduced by truncatingthe adaptive codebook search such that fractional pitch periods are notconsidered within the search (not searching the non-integer lags). Thesemodifications are made to the ITU-T G.729 Annex A, C language routinePitch_fr3_fast( ). The complexity of a G.729 Annex A voice encoder maybe further reduced by substantially reducing the complexity of the fixedexcitation search. The search complexity may be reduced by bypassing thedepth first search 4, phase A: track 3 and 0 search and the depth firstsearch 4, phase B: track 1 and 2 search.

Each modification reduces the computational complexity but alsominimally reduces the resultant voice quality. However, since the voiceencoders are externally managed by the resource manager to minimizeoccasional system resource overloads, the voice encoder shouldpredominately operate with no complexity reductions. The preferredembedded software embodiment should include the standard code as well asthe modifications required to reduce the system complexity. The resourcemanager should preferably minimize power consumption and computationalcycles by invoking complexity reductions which have substantially noimpact on voice quality. The different complexity reductions schemesshould be selected dynamically based on the processing requirements forthe current frame (over all voice channels) and the statistics of thevoice signals on each channel (voice level, voicing, etc).

Although complexity reductions are rare, the appropriate PXDs andassociated services invoked in the network VHDs should preferablyincorporate numerous functional features to accommodate such complexityreductions. For example, the appropriate voice mode PXDs and associatedservices should preferably include a main routine which executes thecomplexity reductions described above with a variety of complexitylevels. For example, various complexity levels may be mandated bysetting various complexity reduction flags. In addition, the resourcemanager should accurately measure the resource requirements of PXDs andservices with fixed resource requirements (i.e. complexity is notcontrollable), to support the computation of peak complexity and averagecomplexity. Also, a function that returns the estimated complexity incycles according to the desired complexity reduction level shouldpreferably be included.

The described exemplary embodiment preferably includes four complexityreduction levels. In the first level, all complexity reductions aredisabled so that the complexity of the PXDs and services is not reduced.

The second level provides minimal or transparent complexity reductions(reductions which should preferably have substantially no observableimpact on performance under most conditions). In the transparent modethe voice encoders (G.729, G.723.1) preferably use voluntary reductionsand the echo canceller is forced into the bypass mode and adaption istoggled (i.e., adaptive is enabled for every other frame). Voluntaryreductions for G.723.1 voice encoders are preferably selected asfollows. First, if the frame energy is less than −55 dBm0, then theadaptive codebook is bypassed and the fixed excitation searches arereduced, as per above. If the frame energy is less than −45 dBm0 butgreater than −55 dBm0, then the adaptive codebook is partially searchedand the fixed excitation searches are reduced as per above. In addition,if the open loop normalized pitch correlation is less than 0.305 thenthe adaptive codebook is partially searched. Otherwise, no complexityreductions are done. Similarly, voluntary reductions for the G.729 voiceencoders preferably proceed as follows: first, if the frame energy isless than −55 dBm0, then the adaptive codebook is bypassed and the fixedexcitation search is reduced per above. Next if the frame energy is lessthan −45 dBm0 but greater than −55 dBm0, then the reduced complexityadaptive codebook is used and the excitation search complexity isreduced. Otherwise, no complexity reduction is used.

The third level of complexity reductions provides minor complexityreductions (reductions which may result in a slight degradation of voicequality or performance). For example, in the third level the voiceencoders preferably use voluntary reductions, “find_best” reduction(G.723.1), fixed codebook threshold change (5.3 kbps G.723.1), open looppitch search reduction (G.723.1 only), and minimal adaptive codebookreduction (G.729 and G.723.1). In addition, the echo canceller is forcedinto the bypass mode and adaption is toggled.

In the fourth level major complexity reductions occur, that isreductions which should noticeably effect the performance quality. Forexample, in the fourth level of complexity reductions the voice encodersuse the same complexity reductions as those used for level threereductions, as well as adding a bypass adaptive codebook reduction(G.729 and G.723.1). In addition, the echo canceller is forced into thebypass mode and adaption is completely disabled. The resource managerpreferably limits the invocation of fourth level major reductions toextreme circumstances, such as, for example when there is double talk onall active channels.

The described exemplary resource manager monitors system resourceutilization. Under normal system operating conditions, complexityreductions are not mandated on the echo canceller or voice encoders.Voice/FAX and data traffic is packetized and transferred in packets. Theecho canceller removes echos, the DTMF detector detects the presence ofkeypad signals, the VAD detects the presence of voice, and the voiceencoders compress the voice traffic into packets. However, when systemresources are overtaxed and complexity reductions are required there areat least two methods for controlling the voice encoder. In the firstmethod, the complexity level for the current frame is estimated from theinformation contained within previous voice frames and from theinformation gained from the echo canceller on the current voice frame.The resource manager then mandates complexity reductions for theprocessing of frames in the current frame interval in accordance withthese estimations.

Alternatively, the voice encoders may be divided into a “front end” anda “back end”. The front end performs voice activity detection and openloop pitch detection (in the case of G.723.1 and G.729 Annex A) on allchannels operating on the DSP. Subsequent to the execution of the frontend function for all channels of a particular voice encoder, the systemcomplexity may be estimated based on the known information. Complexityreductions may then be mandated to ensure that the current processingcycle can satisfy the processing requirements of the voice encoders anddecoders. This alternative method is preferred because the state of theVAD is known whereas in the previously described method the state of theVAD is estimated.

In the alternate method, once the front end processing is complete sothat the state of the VAD and the voicing state for all channels isknown, the system complexity may be estimated based on the knownstatistics for the current frame. In the first method, the state of theVAD and the voicing state may be estimated based on available knowninformation. For example, the echo canceller processes a voice encoderinput signal to remove line echos prior to the activation of the voiceencoder. The echo canceller may estimate the state of the VAD based onthe power level of a reference signal and the voice encoder input signalso that the complexity level of all controllable PXDs and services maybe updated to determine the estimated complexity level of each assumingno complexity reductions have been invoked. If the sum of all thevarious complexity estimates is less than the complexity budget, nocomplexity reductions are required. Otherwise, the complexity level ofall system components are estimated assuming the invocation of thetransparent complexity reduction method to determine the estimatedcomplexity resources required for the current processing frame. If thesum of the complexity estimates with transparent complexity reductionsin place is less than the complexity budget, then the transparentcomplexity reduction is used for that frame. In a similar manner, moreand more severe complexity reduction is considered until systemcomplexity satisfies the prescribed budget.

The operating system should preferably allow processing to exceed thereal-time constraint, i.e. maximum processing capability for theunderlying DSP, in the short term. Thus data that should normally beprocessed within a given time frame or cycle may be buffered andprocessed in the next sequence. However, the overall complexity orprocessor loading must remain (on average) within the real-timeconstraint. This is a tradeoff between delay/jitter and channel density.Since packets may be delayed (due to processing overruns) overall end toend delay may increase slightly to account for the processing jitter.

Referring to FIG. 7, a preferred echo canceller has been modified toinclude an echo canceller bypass switch that invokes an echo suppressorin lieu of echo cancellation under certain system conditions so as toreduce processor loading. In addition, in the described exemplaryembodiment the resource manager may instruct the adaptation logic 136 todisable filter adapter 134 so as to reduce processor loading underreal-time constraints. The system will preferably limit adaptation on afair and equitable basis when processing overruns occur. For example, iffour echo cancellers are adapting when a processing over run occurs, theresource manager may disable the adaption of echo cancellers one andtwo. If the processing over run continues, the resource manger shouldpreferably enable adaption of echo cancellers one and two, and reducesystem complexity by disabling the adaptation of echo cancellers threeand four. This limitation should preferably be adjusted such thatchannels which are fully adapted have adaptation disabled first. In thedescribed exemplary embodiment, the operating systems should preferablycontrol the subfunctions to limit peak system complexity. Thesubfunctions should be co-operative and include modifications to theecho canceller and the speech encoders.

B. The Fax Relay Mode

Fax relay mode provides signal processing of fax signals. As shown inFIG. 20, fax relay mode enables the transmission of fax signals over apacket based system such as VoIP, VoFR, FRF-11, VTOA, or any otherproprietary network. The fax relay mode should also permit data signalsto be carried over traditional media such as TDM. Network gateways 378a, 378 b, 378 c, the operating platform for the signal processing systemin the described exemplary embodiment, support the exchange of faxsignals between a packet based network 376 and various fax machines 380a, 380 b, 380 c. For the purposes of explanation, the first fax machineis a sending fax 380 a. The sending fax 380 a is connected to thesending network gateway 378 a through a PSTN line 374. The sendingnetwork gateway 378 a is connected to a packet based network 376.Additional fax machines 380 b, 380 c are at the other end of the packetbased network 376 and include receiving fax machines 380 b, 380 c andreceiving network gateways 378 b, 378 c. The receiving network gateways378 b, 378 b may provide a direct interface between their respective faxmachines 380 b, 380 c and the packet based network 376.

The transfer of fax signals over packet based networks may beaccomplished by at least three alternative methods. In the first method,fax data signals are exchanged in real time. Typically, the sending andreceiving fax machines are spoofed to allow transmission delays plusjitter of up to about 1.2 seconds. The second, store and forward mode,is a non real time method of transferring fax data signals. Typically,the fax communication is transacted locally, stored into memory andtransmitted to the destination fax machine at a subsequent time. Thethird mode is a combination of store and forward mode with minimalspoofing to provide an approximate emulation of a typical faxconnection.

In the fax relay mode, the network VHD invokes the packet fax dataexchange. The packet fax data exchange provides demodulation andre-modulation of fax data signals. This approach results in considerablebandwidth savings since only the underlying unmodulated data signals aretransmitted across the packet based network. The packet fax dataexchange also provides compensation for network jitter with a jitterbuffer similar to that invoked in the packet voice exchange.Additionally, the packet fax data exchange compensates for lost datapackets with error correction processing. Spoofing may also be providedduring various stages of the procedure between the fax machines to keepthe connection alive.

The packet fax data exchange is divided into two basic functional units,a demodulation system and a re-modulation system. In the demodulationsystem, the network VHD couples fax data signals from a circuit switchednetwork, or a fax machine, to the packet based network. In there-modulation system, the network VHD couples fax data signals from thepacket network to the switched circuit network, or a fax machinedirectly.

During real time relay of fax data signals over a packet based network,the sending and receiving fax machines are spoofed to accommodatenetwork delays plus jitter. Typically, the packet fax data exchange canaccommodate a total delay of up to about 1.2 seconds. Preferably, thepacket fax data exchange supports error correction mode (ECM) relayfunctionality, although a full ECM implementation is typically notrequired. In addition, the packet fax data exchange should preferablypreserve the typical call duration required for a fax session over aPSTN/ISDN when exchanging fax data signals between two terminals.

The packet fax data exchange for the real time exchange of fax datasignals between a circuit switched network and a packet based network isshown schematically in FIG. 21. In this exemplary embodiment, aconnecting PXD (not shown) connecting the fax machine to the switchboard 32′ is transparent, although those skilled in the art willappreciate that various signal conditioning algorithms could beprogrammed into PXD such as echo cancellation and gain.

After the PXD (not shown), the incoming fax data signal 390 a is coupledto the demodulation system of the packet fax data exchange operating inthe network VHD via the switchboard 32′. The incoming fax data signal390 a is received and buffered in an ingress media queue 390. A V.21data pump 392 demodulates incoming T.30 message so that T.30 relay logic394 can decode the received T.30 messages 394 a. Local T.30 indications394 b are packetized by a packetization engine 396 and if required,translated into T.38 packets via a T.38 shim 398 for transmission to aT.38 compliant remote network gateway (not shown) across the packetbased network. The V.21 data pump 392 is selectively enabled/disabled394 c by the T.30 relay logic 394 in accordance with thereception/transmission of the T.30 messages or fax data signals. TheV.21 data pump 392 is common to the demodulation and re-modulationsystem. The V.21 data pump 392 communicates T.30 messages such as forexample called station tone (CED) and calling station tone (CNG) tosupport fax setup between a local fax device (not shown) and a remotefax device (not shown) via the remote network gateway.

The demodulation system further includes a receive fax data pump 400which demodulates the fax data signals during the data transfer phase.The receive fax data pump 400 supports the V.27ter standard for fax datasignal transfer at 2400/4800 bps, the V.29 standard for fax data signaltransfer at 7200/9600 bps, as well as the V.17 standard for fax datasignal transfer at 7200/9600/12000/14400 bps. The V.34 fax standard,once approved, may also be supported. The T.30 relay logic 394enables/disables 394 d the receive fax data pump 400 in accordance withthe reception of the fax data signals or the T.30 messages.

If error correction mode (ECM) is required, receive ECM relay logic 402performs high level data link control (HDLC) de-framing, including bitde-stuffing and preamble removal on ECM frames contained in the datapackets. The resulting fax data signals are then packetized by thepacketization engine 396 and communicated across the packet basednetwork. The T.30 relay logic 394 selectively enables/disables 394 e thereceive ECM relay logic 402 in accordance with the error correction modeof operation.

In the re-modulation system, if required, incoming data packets arefirst translated from a T.38 packet format to a protocol independentformat by the T.38 packet shim 398. The data packets are thende-packetized by a depacketizing engine 406. The data packets maycontain T.30 messages or fax data signals. The T.30 relay logic 394reformats the remote T.30 indications 394 f and forwards the resultingT.30 indications to the V.21 data pump 392. The modulated output of theV.21 data pump 392 is forwarded to an egress media queue 408 fortransmission in either analog format or after suitable conversion, as 64kbps PCM samples to the local fax device over a circuit switchednetwork, such as for example a PSTN line.

De-packetized fax data signals are transferred from the depacketizingengine 406 to a jitter buffer 410. If error correction mode (ECM) isrequired, transmitting ECM relay logic 412 performs HDLC de-framing,including bit stuffing and preamble addition on ECM frames. Thetransmitting ECM relay logic 412 forwards the fax data signals, (in theappropriate format) to a transmit fax data pump 414 which modulates thefax data signals and outputs 8 KHz digital samples to the egress mediaqueue 408. The T.30 relay logic selectively enables/disables (394 g) thetransmit ECM relay logic 412 in accordance with the error correctionmode of operation.

The transmit fax data pump 414 supports the V.27ter standard for faxdata signal transfer at 2400/4800 bps, the V.29 standard for fax datasignal transfer at 7200/9600 bps, as well as the V.17 standard for faxdata signal transfer at 7200/9600/12000/14400 bps. The T.30 relay logicselectively enables/disables (394 h) the transmit fax data pump 414 inaccordance with the transmission of the fax data signals or the T.30message samples.

If the jitter buffer 410 underflows, a buffer low indication 410 a iscoupled to spoofing logic 416. Upon receipt of a buffer low indicationduring the fax data signal transmission, the spoofing logic 416 inserts“spoofed data” at the appropriate place in the fax data signals via thetransmit fax data pump 414 until the jitter buffer 410 is filled to apre-determined level, at which time the fax data signals are transferredout of the jitter buffer 410. Similarly, during the transmission of theT.30 message indications, the spoofing logic 416 can insert “spoofeddata” at the appropriate place in the T.30 message samples via the V.21data pump 392.

1. Data Rate Management

An exemplary embodiment of the packet fax data exchange complies withthe T.38 recommendations for real-time Group 3 facsimile communicationover packet based networks. In accordance with the T.38 standard, thepreferred system should therefore, provide packet fax data exchangesupport at both the T.30 level (see ITU Recommendation T.30—“Proceduresfor Document Facsimile Transmission in the General Switched TelephoneNetwork”, 1988) and the T4 level (see ITU RecommendationT.4—“Standardization of Group 3 Facsimile Apparatus For DocumentTransmission”, 1998), the contents of each of these ITU recommendationsbeing incorporated herein by reference as if set forth in full. Onefunction of the packet fax data exchange is to relay the set up(capabilities) parameters in a timely fashion. Spoofing may be needed ateither or both the T.30 and T.4 levels to maintain the fax session whileset up parameters are negotiated at each of the network gateways andrelayed in the presence of network delays and jitter.

In accordance with the industry T.38 recommendations for real time Group3 communication over packet based networks, the described exemplaryembodiment relays all information including; T.30 preamble indications(flags), T.30 message data, as well as T.30 image data between thenetwork gateways. The T.30 relay logic 394 in the sending and receivingnetwork gateways then negotiate parameters as if connected via a PSTNline. The T.30 relay logic 394 interfaces with the V.21 data pump 392and the receive and transmit data pumps 400 and 414 as well as thepacketization engine 396 and the depacketizing engine 406 to ensure thatthe sending and the receiving fax machines 380(a) and 380(b)successfully and reliably communicate. The T.30 relay logic 394 provideslocal spoofing, using command repeats (CRP), and automatic repeatrequest (ARQ) mechanisms, incorporated into the T.30 protocol, to handledelays associated with the packet based network. In addition, the T.30relay logic 394 intercepts control messages to ensure compatibility ofthe rate negotiation between the near end and far end machines includingHDLC processing, as well as lost packet recovery according to the T.30ECM standard.

FIG. 22 demonstrates message flow over a packet based network between asending fax machine 380 a (see FIG. 20) and the receiving fax device 380b (see FIG. 20) in non-ECM mode. The PSTN fax call is divided into fivephases: call establishment, control and capabilities exchange, pagetransfer, end of page and multi-page signaling and call release. In thecall establishment phase, the sending fax machine dials the sendingnetwork gateway 378 a (see FIG. 20) which forwards calling tone (CNG)(not shown) to the receiving network gateway 378 b (see FIG. 20). Thereceiving network gateway responds by alerting the receiving faxmachine. The receiving fax machine answers the call and sends calledstation (CED) tones. The CED tones are detected by the V.21 data pump392 of the receiving network gateway which issues an event 420indicating the receipt of CED which is then relayed to the sendingnetwork gateway. The sending network gateway forwards the CED tone 422to the sending fax device. In addition, the V.21 data pump of thereceiving network gateway invokes the packet fax data exchange.

In the control and capabilities exchange, the receiving network gatewaytransmits T.30 preamble (HDLC flags) 424 followed by called subscriberidentification (CSI) 426 and digital identification signal (DIS) 428message which contains the capabilities of the receiving fax device. Thesending network gateway, forwards the HDLC flags, CSI and DIS to thesending fax device. Upon receipt of CSI and DIS, the sending fax devicedetermines the conditions for the call by examining its own capabilitiestable relative to those of the receiving fax device. The sending faxdevice issues a command to the sending network gateway 430 to begintransmitting HDLC flags. Next, the sending fax device transmitssubscriber identification (TSI) 432 and digital command signal (DCS) 434messages, which define the conditions of the call to the sending networkgateway. In response, the sending network gateway forwards V.21 HDLCsending subscriber identification/frame check sequences and digitalcommand signal/frame check sequences to the receiving fax device via thereceiving network gateway. Next the sending fax device transmitstraining check (TCF) fields 436 to verify the training and ensure thatthe channel is suitable for transmission at the accepted data rate.

The TCF 436 may be managed by one of two methods. The first method,referred to as the data rate management method one in the T.38 standard,the receiving network gateway locally generate TCF. Confirmation toreceive (CFR) is returned to the sending fax device 380(a), when thesending network gateway receives a confirmation to receive (CFR) 438from the receiving fax machine via the receiving network gateway, andthe TCF training 436 from the sending fax machine is receivedsuccessfully. In the event that the receiving fax machine receives a CFRand the TCF training 436 from the sending fax machine subsequentlyfails, then DCS 434 from the sending fax machine is again relayed to thereceiving fax machine. The TCF training 436 is repeated until anappropriate rate is established which provides successful TCF training436 at both ends of the network.

In a second method to synchronize the data rate, referred to as the datarate management method two in the T.38 standard, the TCF data sequencereceived by the sending network gateway is forwarded from the sendingfax machine to the receiving fax machine via the receiving networkgateway. The sending and receiving fax machines then perform speedselection as if connected via a regular PSTN.

Upon receipt of confirmation to receive (CFR) 440 which indicates thatall capabilities and the modulation speed have been confirmed, thesending fax machine enters the page transfer phase, and transmits imagedata 444 along with its training preamble 442. The sending networkgateway receives the image data and forwards the image data 444 to thereceiving network gateway. The receiving network gateway then sends itsown training preamble 446 followed by the image data 448 to thereceiving fax machine.

In the end of page and multi-page signaling phase, after the page hasbeen successfully transmitted, the sending fax device sends an end ofprocedures (EOP) 450 message if the fax call is complete and all pageshave been transmitted. If only one of multiple pages has beensuccessfully transmitted, the sending fax device transmits a multi-pagesignal (MPS). The receiving fax device responds with messageconfirmation (MCF) 452 to indicate the message has been successfullyreceived and that the receiving fax device is ready to receiveadditional pages. The release phase is the final phase of the call,where at the end of the final page, the receiving fax machine sends amessage confirmation (MCF) 452, which prompts the sending fax machine totransmit a disconnect (DCN) signal 454. The call is then terminated atboth ends of the network.

ECM fax relay message flow is similar to that described above. Allpreambles, messages and page transfers (phase C) HDLC data are relayedthrough the packet based network. Phase C HDLC data is de-stuffed and,along with the preamble and frame checking sequences (FCS), removedbefore being relayed so that only fax image data itself is relayed overthe packet based network. The receiving network gateway performs bitstuffing and reinserts the preamble and FCS.

2. Spoofing Techniques

Spoofing refers to the process by which a facsimile transmission ismaintained in the presence of data packet under-run due to severenetwork jitter or delay. An exemplary embodiment of the packet fax dataexchange complies with the T.38 recommendations for real-time Group 3facsimile communication over packet based networks. In accordance withthe T.38 recommendations, a local and remote T.30 fax device communicateacross a packet based network via signal processing systems, which forthe purposes of explanation are operating in network gateways. Inoperation, each fax device establishes a facsimile connection with itsrespective network gateway in accordance with the ITU-T.30 standards andthe signal processing systems operating in the network gateways relaydata signals across a packet based network.

In accordance with the T.30 protocol, there are ceratin time constraintson the handshaking and image data transmission for the facsimileconnection between the T.30 fax device and its respective networkgateway. The problem that arises is that the T.30 facsimile protocol isnot designed to accommodate the significant jitter and packet delay thatis common to communications across packet based networks. To preventtermination of the fax connection due to severe network jitter or delay,it is, therefore, desirable to ensure that both T.30 fax devices can bespoofed during periods of data packet under-run. FIG. 23 demonstratesfax communication 466 under the T.30 protocol, wherein a handshakenegotiator 468, typically a low speed modem such as V.21, performshandshake negotiation and fax image data is communicated via a highspeed data pump 470 such as V.27, V.29 or V.17. In addition, fax imagedata can be transmitted in an error correction mode (ECM) 472 or nonerror correction mode (non-ECM) 474, each of which uses a different dataformat.

Therefore, in the described exemplary embodiment, the particularspoofing technique utilized is a function of the transmission format. Inthe described exemplary embodiment, HDLC preamble 476 is used to spoofthe T.30 fax devices during V.21 handshaking and during transmission offax image data in the error correction mode. However, zero-bit filling478 is used to spoof the T.30 fax devices during fax image data transferin the non error correction mode. Although fax relay spoofing isdescribed in the context of a signal processing system with the packetdata fax exchange invoked, those skilled in the art will appreciate thatthe described exemplary fax relay spoofing method is likewise suitablefor various other telephony and telecommunications application.Accordingly, the described exemplary embodiment of fax relay spoofing ina signal processing system is by way of example only and not by way oflimitation.

a. V.21 HDLC Preamble Spoofing

The T.30 relay logic 394 packages each message or command into a HDLCframe which includes preamble flags. An HDLC frame structure is utilizedfor all binary-coded V.21 facsimile control procedures. The basic HDLCstructure consists of a number of frames, each of which is subdividedinto a number of fields. The HDLC frame structure provides for framelabeling and error checking. When a new facsimile transmission isinitiated, HDLC preamble in the form of synchronization sequences aretransmitted prior to the binary coded information. The HDLC preamble isV.21 modulated bit streams of “01111110 (0x7e)”.

In the described exemplary embodiment, spoofing techniques are utilizedat the T.30 and T.4 levels to manage extended network delays and jitter.Turning back to FIG. 21, the T.30 relay logic 394 waits for a responseto any message or command transmitted across the packet based networkbefore continuing to the next state or phase. In accordance with anexemplary spoofing technique, the sending and receiving network gateways378 a, 378 b (See FIG. 20) spoof their respective fax machines 380 a,380 b by locally transmitting HDLC preamble flags if a response to atransmitted message is not received from the packet based network withinapproximately 1.5-2.0 seconds. The maximum length of the preamble islimited to about four seconds. If a response from the packet basednetwork arrives before the spoofing time out, each network gatewayshould preferably transmit a response message to its respective faxmachine following the preamble flags. Otherwise, if the network responseto a transmitted message is not received prior to the spoofing time out(in the range of about 5.5-6.0 seconds), the response is assumed to belost. In this case, when the network gateway times out and terminatespreamble spoofing, the local fax device transmits the message commandagain. Each network gateway repeats the spoofing technique until asuccessful handshake is completed or its respective fax machinedisconnects.

b. ECM HDLC Preamble Spoofing

The packet fax data exchange utilizes an HDLC frame structure for ECMhigh-speed data transmission. Preferably, the frame image data isdivided by one or more HDLC preamble flags. If the network under-runsdue to jitter or packet delay, the network gateways spoof theirrespective fax devices at the T.4 level by adding extra HDLC flagsbetween frames. This spoofing technique increases the sending time tocompensate for packet under-run due to network jitter and delay.Returning to FIG. 21 if the jitter buffer 410 underflows, a buffer lowindication 410 a is coupled to the spoofing logic 416. Upon receipt of abuffer low indication during the fax data signal transmission, thespoofing logic 416 inserts HDLC preamble flags at the frame boundary viathe transmit fax data pump 414. When the jitter buffer 410 is filled toa pre-determined level, the fax image data is transferred out of thejitter buffer 410.

In the described exemplary embodiment, the jitter buffer 410 must besized to store at least one HDLC frame so that a frame boundary may belocated. The length of the largest T.4 ECM HDLC frame is 260 octets or130 16-bit words. Spoofing is preferably activated when the number ofpackets stored in the jitter buffer 410 drops to a predeterminedthreshold level. When spoofing is required, the spoofing logic 416 addsHDLC flags at the frame boundary as a complete frame is beingreassembled and forwarded to the transmit fax data pump 414. Thiscontinues until the number of data packets in the jitter buffer 410exceeds the threshold level. The maximum time the network gateways willspoof their respective local fax devices can vary but can generally beabout ten seconds.

c. Non-ECM Spoofing with Zero Bit Filling

T.4 spoofing handles delay impairments during page transfer or C phaseof a fax call. For those systems that do not utilize ECM, phase Csignals comprise a series of coded image data followed by fill bits andend-of-line (EOL) sequences. Typically, fill bits are zeros insertedbetween the fax data signals and the EOL sequences, “000000000001”. Fillbits ensure that a fax machine has time to perform the variousmechanical overhead functions associated with any line it receives. Fillbits can also be utilized to spoof the jitter buffer to ensurecompliance with the minimum transmission time of the total coded scanline established in the pre-message V.21 control procedure. The numberof the bits of coded image contained in the data signals associated withthe scan line and transmission speed limit the number of fill bits thatcan be added to the data signals. Preferably, the maximum transmissionof any coded scan line is limited to less than about 5 sec. Thus, if thecoded image for a given scan line contains 1000 bits and thetransmission rate is 2400 bps, then the maximum duration of fill time is(5−(1000+12)/2400)=4.57 sec.

Generally, the packet fax data exchange utilizes spoofing if the networkjitter delay exceeds the delay capability of the jitter buffer 410. Inaccordance with the EOL spoofing method, fill bits can only be insertedimmediately before an EOL sequence, so that the jitter buffer 410 shouldpreferably store at least one EOL sequence. Thus the jitter buffer 410should preferably be sized to hold at least one entire scan line of datato ensure the presence of at least one EOL sequence within the jitterbuffer 410. Thus, depending upon transmission rate, the size of thejitter buffer 410 can become prohibitively large. The table belowsummarizes the desired jitter buffer data space to perform EOL spoofingfor various scan line lengths. The table assumes that each pixel isrepresented by a single bit. The values represent an approximate upperlimit on the required data space, but not the absolute upper limit,because in theory at least, the longest scan line can consist ofalternating black and white pixels which would require an average of 4.5bits to represent each pixel rather than the one to one ratio summarizedin the table.

Scan Number sec to Line of print out sec to print sec to print sec toprint Length words at 2400 out at 4800 out at 9600 out at 14400 1728 1080.72 0.36 0.18 0.12 2048 128 0.853 0.427 0.213 0.14 2432 152 1.01 0.5070.253 0.17 3456 216 1.44 0.72 0.36 0.24 4096 256 2 0.853 0.43 0.28 4864304 2.375 1.013 0.51 0.34

To ensure the jitter buffer 410 stores an EOL sequence, the spoofinglogic 416 should be activated when the number of data packets stored inthe jitter buffer 410 drops to a threshold level. Typically, a thresholdvalue of about 200 msec is used to support the most commonly used faxsetting, namely a fax speed of 9600 bps and scan line length of 1728. Analternate spoofing method should be used if an EOL sequence is notcontained within the jitter buffer 410, otherwise the call will have tobe terminated. An alternate spoofing method uses zero run length codewords. This method requires real time image data decoding so that theword boundary is known. Advantageously, this alternate method reducesthe required size of the jitter buffer 410.

Simply increasing the storage capacity of the jitter buffer 410 canminimize the need for spoofing. However, overall network delay increaseswhen the size of the jitter buffer 410 is increased. Increased networkdelay may complicate the T.30 negotiation at the end of page or end ofdocument, because of susceptibility to time out. Such a situation ariseswhen the sending fax machine completes the transmission of high speeddata, and switches to an HDLC phase and sends the first V.21 packet inthe end of page/multi-page signaling phase, (i.e. phase D). The sendingfax machine must be kept alive until the response to the V.21 datapacket is received. The receiving fax device requires more time to flusha large jitter buffer and then respond, hence complicating the T.30negotiation.

In addition, the length of time a fax machine can be spoofed is limited,so that the jitter buffer 410 can not be arbitrarily large. A pipelinestore and forward relay is a combination of store and forward andspoofing techniques to approximate the performance of a typical Group 3fax connection when the network delay is large (on the order of secondsor more). One approach is to store and forward a single page at a time.However, this approach requires a significant amount of memory (10Kwords or more). One approach to reduce the amount of memory requiredentails discarding scan lines on the sending network gateway andperforming line repetition on the receiving network gateway so as tomaintain image aspect ratio and quality. Alternatively, a partial pagecan be stored and forwarded thereby reducing the required amount ofmemory.

The sending and receiving fax machines will have some minimaldifferences in clock frequency. ITU standards recommends a data pumpdata rate of ±100 ppm, so that the clock frequencies between thereceiving and sending fax machines could differ by up to 200 ppm.Therefore, the data rate at the receiving network gateway (jitter buffer410) can build up or deplete at a rate of 1 word for every 5000 wordsreceived. Typically a fax page is less than 1000 words so that end toend clock synchronization is not required.

C. Data Relay Mode

Data relay mode provides full duplex signal processing of data signals.As shown in FIG. 24, data relay mode enables the transmission of datasignals over a packet based system such as VoIP, VoFR, FRF-11, VTOA, orany other proprietary network. The data relay mode should also permitdata signals to be carried over traditional media such as TDM. Networkgateways 496 a, 496 b, 496 c, support the exchange of data signalsbetween a packet based network 494 and various data modems 492 a, 492 b,492 c. For the purposes of explanation, the first modem is referred toas a call modem 492 a. The call modem 492 a is connected to the callnetwork gateway 496 a through a PSTN line. The call network gateway 496a is connected to a packet based network 494. Additional modems 492 b,492 c are at the other end of the packet based network 494 and includeanswer modems 492 b, 492 c and answer network gateways 496 b, 496 c. Theanswer network gateways 496 b, 496 c provide a direct interface betweentheir respective modems 492 b, 492 c and the packet based network 494.

In data relay mode, a local modem connection is established on each endof the packet based network 494. That is, the call modem 492 a and thecall network gateway 496 a establish a local modem connection, as doesthe destination answer modem 492 b and its respective answer networkgateway 496 b. Next, data signals are relayed across the packet basednetwork 494. The call network gateway 496 a demodulates the data signaland formats the demodulated data signal for the particular packet basednetwork 494. The answer network gateway 496 b compensates for networkimpairments and remodulates the encoded data in a format suitable forthe destination answer modem 492 b. This approach results inconsiderable bandwidth savings since only the underlying demodulateddata signals are transmitted across the packet based network.

In the data relay mode, the packet data modem exchange providesdemodulation and modulation of data signals. With full duplexcapability, both modulation and demodulation of data signals can beperformed simultaneously. The packet data modem exchange also providescompensation for network jitter with a jitter buffer similar to thatinvoked in the packet voice exchange. Additionally, the packet datamodem exchange compensates for system clock jitter between modems with adynamic phase adjustment and resampling mechanism. Spoofing may also beprovided during various stages of the call negotiation procedure betweenthe modems to keep the connection alive.

The packet data modem exchange invoked by the network VHD in the datarelay mode is shown schematically in FIG. 25. In the described exemplaryembodiment, a connecting PXD (not shown) connecting a modem to theswitch board 32′ is transparent, although those skilled in the art willappreciate that various signal conditioning algorithms could beprogrammed into the PXD such as filtering, echo cancellation and gain.

After the PXD, the data signals are coupled to the network VHD via theswitchboard 32′. The packet data modem exchange provides two waycommunication between a circuit switched network and packet basednetwork with two basic functional units, a demodulation system and aremodulation system. In the demodulation system, the network VHDexchanges data signals from a circuit switched network, or a telephonydevice directly, to a packet based network. In the remodulation system,the network VHD exchanges data signals from the packet based network tothe PSTN line, or the telephony device.

In the demodulation system, the data signals are received and bufferedin an ingress media queue 500. A data pump receiver 504 demodulates thedata signals from the ingress media queue 500. The data pump receiver504 supports the V.22bis standard for the demodulation of data signalsat 1200/2400 bps; the V.32bis standard for the demodulation of datasignals at 4800/7200/9600/12000/14400 bps, as well as the V.34 standardfor the demodulation of data signals up to 33600 bps. Moreover, the V.90standard may also be supported. The demodulated data signals are thenpacketized by the packetization engine 506 and transmitted across thepacket based network.

In the remodulation system, packets of data signals from the packetbased network are first depacketized by a depacketizing engine 508 andstored in a jitter buffer 510. A data pump transmitter 512 modulates thebuffered data signals with a voiceband carrier. The modulated datasignals are in turn stored in the egress media queue 514 before beingoutput to the PXD (not shown) via the switchboard 32′. The data pumptransmitter 512 supports the V.22bis standard for the transfer of datasignals at 1200/2400 bps; the V.32bis standard for the transfer of datasignals at 4800/7200/9600/12000/14400 bps, as well as the V.34 standardfor the transfer of data signal up to 33600 bps. Moreover, the V.90standard may also be supported.

During jitter buffer underflow, the jitter buffer 510 sends a buffer lowindication 510 a to spoofing logic 516. When the spoofing logic 516receives the buffer low signal indicating that the jitter buffer 510 isoperating below a predetermined threshold level, it inserts spoofed dataat the appropriate place in the data signal via the data pumptransmitter 512. Spoofing continues until the jitter buffer 510 isfilled to the predetermined threshold level, at which time data signalsare again transferred from the jitter buffer 510 to the data pumptransmitter 512.

End to end clock logic 518 also monitors the state of the jitter buffer510. The clock logic 518 controls the data transmission rate of the datapump transmitter 512 in correspondence to the state of the jitter buffer510. When the jitter buffer 510 is below a predetermined thresholdlevel, the clock logic 518 reduces the transmission rate of the datapump transmitter 512. Likewise, when the jitter buffer 510 is above apredetermined threshold level, the clock logic 518 increases thetransmission rate of the data pump transmitter 512.

Before the transmission of data signals across the packet based network,the connection between the two modems must first be negotiated through ahandshaking sequence. This entails a two-step process. First, a callnegotiator 502 determines the type of modem (i.e., V.22, V.32bis, V.34,V.90, etc.) connected to each end of the packet based network. Second, arate negotiator 520 negotiates the data signal transmission rate betweenthe two modems.

The call negotiator 502 determines the type of modem connected locally,as well as the type of modem connected remotely via the packet basednetwork. The call negotiator 502 utilizes V.25 automatic answeringprocedures and V.8 auto-baud software to automatically detect modemcapability. The call negotiator 502 receives protocol indication signals502 a (ANSam and V.8 menus) from the ingress media queue 500, as well asAA, AC and other message indications 502 b from the local modem via adata pump state machine 522, to determine the type of modem in uselocally. The call negotiator 502 relays the ANSam answer tones and otherindications 502 e from the data pump state machine 522 to the remotemodem via a packetization engine 506. The call negotiator also receivesANSam, AA, AC and other indications 502 c from a remote modem (notshown) located on the opposite end of the packet based network via adepacketizing engine 508. The call negotiator 502 relays ANSam answertones and other indications 502 d to a local modem (not shown) via anegress media queue 514 of the modulation system. With the ANSam, AA, ACand other indications from the local and remote modems, the callnegotiator 502 can then negotiate a common standard (i.e., V.22,V.32bis, V.34, V.90, etc.) in which the data pumps must communicate withthe local modem and the remote modems.

The packet data modem exchange preferably utilizes indication packets asa means for communicating answer tones, AA, AC and other indicationsignals across the packet based network However, the packet data modemexchange supports data pumps such as V.22bis and V.32bis which do notinclude a well defined error recovery mechanism, so that the modemconnection may be terminated whenever indication packets are lost.Therefore, either the packet data modem exchange or the applicationlayer should ensure proper delivery of indication packets when operatingin a network environment that does not guarantee packet delivery.

The packet data modem exchange can ensure delivery of the indicationpackets by periodically retransmitting the indication packet until someexpected packets are received. For example, in V.32bis relay, the callnegotiator operating under the packet data modem exchange on the answernetwork gateway periodically retransmits ANSam answer tones from theanswer modem to the call modem, until the calling modem connects to theline and transmits carrier state AA.

Alternatively, the packetization engine can embed the indicationinformation directly into the packet header. In this approach, analternate packet format is utilized to include the indicationinformation. During modem handshaking, indication packets transmittedacross the packet based network include the indication information, sothat the system does not rely on the successful transmission ofindividual indication packets. Rather, if a given packet is lost, thenext arriving packet contains the indication information in the packetheader. Both methods increase the traffic across the network. However,it is preferable to periodically retransmit the indication packetsbecause it has less of a detrimental impact on network traffic.

A rate negotiator 520 synchronizes the connection rates at the networkgateways 496 a, 496 b, 496 c (see FIG. 24). The rate negotiator receivesrate control codes 520 a from the local modem via the data pump statemachine 522 and rate control codes 520 b from the remote modem via thedepacketizing engine 508. The rate negotiator 520 also forwards theremote rate control codes 520 a received from the remote modem to thelocal modem via commands sent to the data pump state machine 522. Therate negotiator 520 forwards the local rate control codes 520 c receivedfrom the local modem to the remote modem via the packetization engine506. Based on the exchanged rate codes the rate negotiator 520establishes a common data rate between the calling and answering modems.During the data rate exchange procedure, the jitter buffer 510 should bedisabled by the rate negotiator 520 to prevent data transmission betweenthe call and answer modems until the data rates are successfullynegotiated.

Similarly error control (V.42) and data compression (V.42bis) modesshould be synchronized at each end of the packet based network. Errorcontrol logic 524 receives local error control messages 524 a from thedata pump receiver 504 and forwards those V.14/V.42 negotiation messages524 c to the remote modem via the packetization engine 506. In addition,error control logic 524 receives remote V.14/V.42 indications 524 b fromthe depacketizing engine 508 and forwards those V.14/V.42 indications524 d to the local modem. With the V.14/V.42 indications from the localand remote modems, the error control logic 524 can negotiate a commonstandard to ensure that the network gateways utilize a common errorprotocol. In addition, error control logic 524, communicates thenegotiated error control protocol 524(e) to the spoofing logic 516 toensure data mode spoofing is in accordance with the negotiated errorcontrol mode.

V.42 is a standard error correction technique using advanced cyclicalredundancy checks and the principle of automatic repeat requests (ARQ).In accordance with the V.42 standard, transmitted data signals aregrouped into blocks and cyclical redundancy calculations add errorchecking words to the transmitted data signal stream. The receivingmodem calculates new error check information for the data signal blockand compares the calculated information to the received error checkinformation. If the codes match, the received data signals are valid andanother transfer takes place. If the codes do not match, a transmissionerror has occurred and the receiving modem requests a repeat of the lastdata block. This repeat cycle continues until the entire data block hasbeen received without error.

Various voiceband data modem standards exist for error correction anddata compression. V.42bis and MNP5 are examples of data compressionstandards. The handshaking sequence for every modem standard isdifferent so that the packet data modem exchange should support numerousdata transmission standards as well as numerous error correction anddata compression techniques.

1. End to End Clock Logic

Slight differences in the clock frequency of the call modem and theanswer modem are expected, since the baud rate tolerance for a typicalmodem data pump is ±100 ppm. This tolerance corresponds to a relativelylow depletion or build up rate of 1 in 5000 words. However, the lengthof a modem session can be very long, so that uncorrected difference inclock frequency may result in jitter buffer underflow or overflow.

In the described exemplary embodiment, the clock logic synchronizes thetransmit clock of the data pump transmitter 512 to the average rate atwhich data packets arrive at the jitter buffer 510. The data pumptransmitter 512 packages the data signals from the jitter buffer 510 inframes of data signals for demodulation and transmission to the egressmedia queue 514. At the beginning of each frame of data signals, thedata pump transmitter 512 examines the egress media queue 514 todetermine the remaining buffer space, and in accordance therewith, thedata pump transmitter 512 modulates that number of digital data samplesrequired to produce a total of slightly more or slightly less than 80samples per frame, assuming that the data pump transmitter 512 isinvoked once every 10 msec. The data pump transmitter 512 graduallyadjusts the number of samples per frame to allow the receiving modem toadjust to the timing change. Typically, the data pump transmitter 512uses an adjustment rate of about one ppm per frame. The maximumadjustment should be less than about 200 ppm.

In the described exemplary embodiment, end to end clock logic 518monitors the space available within the jitter buffer 510 and utilizeswater marks to determine whether the data rate of the data pumptransmitter 512 should be adjusted. Network jitter may cause timingadjustments to be made. However, this should not adversely affect thedata pump receiver of the answering modem as these timing adjustmentsare made very gradually.

2. Modem Connection Handshaking Sequence.

a. Call Negotiation.

A single industry standard for the transmission of modem data over apacket based network does not exist. However, numerous common standardsexist for transmission of modem data at various data rates over thePSTN. For example, V.22 is a common standard used to define operation of1200 bps modems. Data rates as high as 2400 bps can be implemented withthe V.22bis standard (the suffix “bis” indicates that the standard is anadaptation of an existing standard). The V.22bis standard groups datasignals into four bit words which are transmitted at 600 baud. The V.32standard supports full duplex, data rates of up to 9600 bps over thePSTN. A V.32 modem groups data signals into four bit words and transmitsat 2400 baud. The V.32bis standard supports duplex modems operating atdata rates up to 14,400 bps on the PSTN. In addition, the V.34 standardsupports data rates up to 33,600 bps on the public switched telephonenetwork. In the described exemplary embodiment, these standards can beused for data signal transmission over the packet based network with acall negotiator that supports each standard.

b. Rate Negotiation.

Rate negotiation refers to the process by which two telephony devicesare connected at the same data rate prior to data transmission. In thecontext of a modem connection in accordance with an exemplary embodimentof the present invention, each modem is coupled to a signal processingsystem, which for the purposes of explanation is operating in a networkgateway, either directly or through a PSTN line. In operation, eachmodem establishes a modem connection with its respective networkgateway, at which point, the modems begin relaying data signals across apacket based network. The problem that arises is that each modem maynegotiate a different data rate with its respective network gateway,depending on the line conditions and user settings. In this instance,the data signals transmitted from one of the modems will enter thepacket based network faster than it can be extracted at the other end bythe other modem. The resulting overflow of data signals may result in alost connection between the two modems. To prevent data signal overflow,it is, therefore, desirable to ensure that both modems negotiate to thesame data rate. A rate negotiator can be used for this purpose. Althoughthe the rate negotiator is described in the context of a signalprocessing system with the packet data modem exchange invoked, thoseskilled in the art will appreciate that the rate negotiator is likewisesuitable for various other telephony and telecommunications application.Accordingly, the described exemplary embodiment of the rate negotiatorin a signal processing system is by way of example only and not by wayof limitation.

In an exemplary embodiment, data rate negotiation is achieved through adata rate negotiation procedure, wherein a call modem independentlynegotiates a data rate with a call network gateway, and an answer modemindependently negotiates a data rate with an answer network gateway. Thecalling and answer network gateways, each having a signal processingsystem running a packet exchange, then exchange data packets containinginformation on the independently negotiated data rates. If theindependently negotiated data rates are the same, then each ratenegotiator will enable its respective network gateway and datatransmission between the call and answer modems will commence.Conversely, if the independently negotiated data rates are different,the rate negotiator will renegotiate the data rate by adopting thelowest of the two data rates. The call and answer modems will thenundergo retraining or rate renegotiation procedures by their respectivenetwork gateways to establish a new connection at the renegotiated datarate. The advantage of this approach is that the data rate negotiationprocedure takes advantage of existing modem functionality, namely, theretraining and rate renegotiation mechanism, and puts it to alternativeusage. Moreover, by retraining both the call and answer modem (one modemwill already be set to the renegotiated rate) both modems areautomatically prevented from sending data.

Alternatively, the calling and answer modems can directly negotiate thedata rate. This method is not preferred for modems with time constrainedhandshaking sequences such as, for example, modems operating inaccordance with the V.22bis or the V.32bis standards. The round tripdelay accommodated by these standards could cause the modem connectionto be lost due to timeout. Instead, retrain or rate renegotiation shouldbe used for data signals transferred in accordance with the V.22bis andV.32bis standards, whereas direct negotiation of the data rate by thelocal and remote modems can be used for data exchange in accordance withthe V.34 and V.90 (a digital modem and analog modem pair for use on PSTNlines at data rates up to 56,000 bps downstream and 33,600 upstream)standards.

c. Exemplary Handshaking Sequences.

(V.22 Handshaking Sequence)

The call negotiator on the answer network gateway, differentiatesbetween modem types and relays the ANSam answer tone. The answer modemtransmits unscrambled binary ones signal (USB1) indications to theanswer mode gateway. The answer network gateway forwards USB1 signalindications to the call network gateway. The call negotiator in the callnetwork gateway assumes operation in accordance with the V.22bisstandard as a result of the USB1 signal indication and terminates thecall negotiator. The packet data modem exchange, in the answer networkgateway then invokes operation in accordance with the V.22bis standardafter an answer tone timeout period and terminates its call negotiator.

V.22bis handshaking does not utilize rate messages or signaling toindicate the selected bit rate as with most high data rate pumps.Rather, the inclusion of a fixed duration signal (S1) indicates that2400 bps operation is to be used. The absence of the S1 signal indicatesthat 1200 bps should be selected. The duration of the S1 signal istypically about 100 msec, making it likely that the call modem willperform rate determination (assuming that it selects 2400 bps) beforerate indication from the answer modem arrives. Therefore, the ratenegotiator in the call network gateway should select 2400 bps operationand proceed with the handshaking procedure. If the answer modem islimited to a 1200 bps connection, rate renegotiation is typically usedto change the operational data rate of the call modem to 1200 bps.Alternatively, if the call modem selects 1200 bps, rate renegotiationwould not be required.

(V.32bis Handshaking Sequence)

V32bis handshaking utilizes rate signals (messages) to specify the bitrate. A relay sequence in accordance with the V.32bis standard is shownin FIG. 26 and begins with the call negotiator in the answer networkgateway relaying ANSam 530 answer tone from the answer modem to the callmodem. After receiving the answer tone for a period of at least onesecond, the call modem connects to the line and repetitively transmitscarrier state A 532. When the call network gateway detects the repeatedtransmission of carrier state A (“AA”), the call network gateway relaysthis information 534 to the answer network gateway. In response theanswer network gateway forwards the AA indication to the answer modemand invokes operation in accordance with the V.32bis standard. Theanswer modem then transmits alternating carrier states A and C 536 tothe answer network gateway. If the answer network gateway receives ACfrom the answer modem, the answer network gateway relays AC 538 to thecall network gateway, thereby establishing operation in accordance withthe V.32bis standard, allowing call negotiator in the call networkgateway to be terminated. Next, data rate alignment is achieved byeither of two methods.

In the first method for data rate alignment of a V.32bis relayconnection, the call modem and the answer modem independently negotiatea data rate with their respective network gateways at each end of thenetwork 540 and 542. Next, each network gateway forwards a connectiondata rate indication 544 and 546 to the other network gateway. Eachnetwork gateway compares the far end data rate to its own data rate. Thepreferred rate is the minimum of the two rates. Rate renegotiation 548and 550 is invoked if the connection rate of either network gateway toits respective modem differs from the preferred rate.

In the second method, rate signals R1, R2 and R3, are relayed to achievedata rate negotiation. FIG. 27 shows a relay sequence in accordance withthe V.32bis standard for this alternate method of rate negotiation. Thecall negotiator relays the answer tone (ANSam) 552 from the answer modemto the call modern. When the call modem detects answer tone, itrepetitively transmits carrier state A 554 to the call network gateway.The call network gateway relays this information (AA) 556 to the answernetwork gateway. The answer network gateway sends the AA 558 to theanswer modem and initiates normal range tone exchange with the answermodem. The answer network gateway then forwards AC 560 to call networkgateway which in turn relays this information 562 to the call modem toinitiate normal range tone exchange between the call network gateway andthe call modem.

The answer modem sends its first training sequence 564 followed by R1(the data rates currently available in the answer modem) to the ratenegotiator in the answer network gateway. When the answer networkgateway receives an R1 indication, it forwards R1 566 to the callnetwork gateway. The answer network gateway then repetitively sendstraining sequences to the answer modem. The call network gatewayforwards the R1 indication 570 of the answer modem to the call modem.The call modem sends training sequences to the call network gateway 572.The call network gateway determines the data rate capability of the callmodem, and forwards the data rate capabilities of the call modem to theanswer network gateway in a data rate signal format. The call modem alsosends an R2 indication 568 (data rate capability of the call modem,preferably excluding rates not included in the previously received R1signal, i.e. not supported by the answer modem) to the call networkgateway which forwards it to the answer network gateway. The callnetwork gateway then repetitively sends training sequences to the callmodem until receiving an R3 signal 574 from the answer modem via theanswer network gateway.

The answer network gateway performs a logical AND operation on the R1signal from the answer modem (data rate capability of the answer modem),the R2 signal from the call modem (data rate capability of the callmodem, excluding rates not supported by the answer modem) and thetraining sequences of the call network gateway (data rate capability ofthe call modem) to create a second rate signal R2 576, which isforwarded to the answer modem. The answer modem sends its secondtraining sequence followed an R3 signal, which indicates the data rateto be used by both modems. The answer network gateway relays R3 574 tothe call network gateway which forwards it to the call modem and beginsoperating at the R3 specified bit rate. However, this method of ratesynchronization is not preferred for V.32bis due to time constrainedhandshaking.

(V.34 Handshaking Sequence)

Data transmission in accordance with the V.34 standard utilizes amodulation parameter (MP) sequence to exchange information pertaining todata rate capability. The MP sequences can be exchanged end to end toachieve data rate synchronization. Initially, the call negotiator in theanswer network gateway relays the answer tone (ANSam) from the answermodem to the call modem. When the call modem receives answer tone, itgenerates a CM indication and forwards it to the call network gateway.When the call network gateway receives a CM indication, it forwards itto the answer network gateway which then communicates the CM indicationwith the answer modem. The answer modem then responds by transmitting aJM sequence to the answer network gateway, which is relayed by theanswer network gateway to the call modem via the call network gateway.If the call network gateway then receives a CJ sequence from the callmodem, the call negotiator in the call network gateway, initiatesoperation in accordance with the V.34 standard, and forwards a CJsequence to the answer network gateway. If the JM menu calls for V.34,the call negotiator in the answer network gateway initiates operation inaccordance with the V.34 standard and the call negotiator is terminated.If a standard other than V.34 is called for, the appropriate procedureis invoked, such as those described previously for V.22 or V.32bis.Next, data rate alignment is achieved by either of two methods.

In a first method for data rate alignment after a V.34 relay connectionis established, the call modem and the answer modem freely negotiate adata rate at each end of the network with their respective networkgateways. Each network gateway forwards a connection rate indication tothe other gateway. Each gateway compares the far end bit rate to therate transmitted by each gateway. For example, the call network gatewaycompares the data rate indication received from the answer modem gatewayto that which it negotiated freely negotiated to with the call modem.The preferred rate is the minimum of the two rates. Rate renegotiationis invoked if the connection rate at the calling or receiving enddiffers from the preferred rate, to force the connection to the desiredrate.

In an alternate method for V.34 rate synchronization, MP sequences areutilized to achieve rate synchronization without rate renegotiation. Thecall modem and the answer modem independently negotiate with the callnetwork gateway and the answer network gateway respectively until phaseIV of the negotiations is reached. The call network gateway and theanswer network gateway exchange training results in the form of MPsequences when Phase IV of the independent negotiations is reached toestablish the primary and auxiliary data rates. The call network gatewayand the answer network gateway are preferably prevented from relaying MPsequences to the call modem and the answer modem respectively until thetraining results for both network gateways and the MP sequences for bothmodems are available. If symmetric rate is enforced, the maximum answerdata rate and the maximum call data rate of the four MP sequences arecompared. The lower data rate of the two maximum rates is the preferreddata rate. Each network gateway sends the MP sequence with the preferredrate to its respective modem so that the calling and answer modemsoperate at the preferred data rate.

If asymmetric rates are supported, then the preferred call-answer datarate is the lesser of the two highest call-answer rates of the four MPsequences. Similarly, the preferred answer-call data rate is the lesserof the two highest answer-call rates of the four MP sequences. Data ratecapabilities may also need to be modified when the MP sequence areformed so as to be sent to the calling and answer modems. The MPsequence sent to the calling and answer modems, is the logical AND ofthe data rate capabilities from the four MP sequences.

(V.90 Handshaking Sequence)

The V.90 standard utilizes a digital and analog modem pair to transmitmodem data over the PSTN line. The V.90 standard utilizes MP sequencesto convey training results from a digital to an analog modem, and asimilar sequence, using constellation parameters (CP) to convey trainingresults from an analog to a digital modem. Under the V.90 standard, thetimeout period is 15 seconds compared to a timeout period of 30 secondsunder the V.34 standard. In addition, the analog modems control thehandshake timing during training. In an exemplary embodiment, the callmodem and the answer modem are the V.90 analog modems. As such the callmodem and the answer modem are beyond the control of the networkgateways during training. The digital modems only control the timingduring transmission of TRN1d, which the digital modern in the networkgateway uses to train its echo canceller.

When operating in accordance with the V.90 standard, the call negotiatorutilizes the V.8 recommendations for initial negotiation. Thus, theinitial negotiation of the V.90 relay session is substantially the sameas the relay sequence described for V.34 rate synchronization method oneand method two with asymmetric rate operation. There are twoconfigurations where V.90 relay may be used. The first configuration isdata relay between two V.90 analog modems, i.e. each of the networkgateways are configured as V.90 digital modems. The upstream ratebetween two V.90 analog modems, according to the V.90 standard, islimited to 33,600 bps. Thus, the maximum data rate for an analog toanalog relay is 33,600 bps. In accordance with the V.90 standard, theminimum data rate a V.90 digital modem will support is 28,800 bps.Therefore, the connection must be terminated if the maximum data ratefor one or both of the upstream directions is less than 28,800 bps, andone or both the downstream direction is in V.90 digital mode. Therefore,the V.34 protocol is preferred over V.90 for data transmission betweenlocal and remote analog modems.

A second configuration is a connection between a V.90 analog modem and aV.90 digital modem. A typical example of such a configuration is when auser within a packet based PABX system dials out into a remote accessserver (RAS) or an Internet service provider (ISP) that uses a centralsite modem for physical access that is V.90 capable. The connection fromPABX to the central site modem may be either through PSTN or directlythrough an ISDN, T1 or E1 interface. Thus the V.90 embodiment shouldpreferably support an analog modem interfacing directly to ISDN, T1 orE1.

For an analog to digital modem connection, the connections at both endsof the packet based network should be either digital or analog toachieve proper rate synchronization. The analog modem decides whether toselect digital mode as specified in INFO1a, so that INFO1a should berelayed between the calling and answer modem via their respectivenetwork gateways before operation mode is synchronized.

Upon receipt of an INFO1a signal from the answer modem, the answernetwork gateway performs a line probe on the signal received from theanswer modem to determine whether digital mode can be used. The callnetwork gateway receives an INFO1a signal from the call modem. The callnetwork gateway sends a mode indication to the answer network gatewayindicating whether digital or analog will be used and initiatesoperation in the mode specified in INFO1a. Upon receipt of an analogmode indication signal from the call network gateway, the answer networkgateway sends an INFO1a sequence to the answer modem. The answer networkgateway then proceeds with analog mode operation. Similarly, if digitalmode is indicated and digital mode can be supported by the answer modem,the answer network gateway sends an INFO1a sequence to the answer modemindicating that digital mode is desired and proceeds with digital modeoperation.

Alternatively, if digital mode is indicated and digital mode can not besupported by the answer modem, the call modem should preferably beforced into analog mode by one of three alternate methods. First, somecommercially available V.90 analog modems may revert to analog modeafter several retrains. Thus, one method to force the call modem intoanalog mode is to force retrains until the call modem selects analogmode operation. In an alternate method, the call network gatewaymodifies its line probe so as to force the call modern to select analogmode. In a third method, the call modem and the answer modem operate indifferent modes. Under this method if the answer modem can not support a28,800 bps data rate the connection is terminated.

3. Data Mode Spoofing

The jitter buffer 510 may underflow during long delays of data signalpackets. Jitter buffer underflow can cause the data pump transmitter 512to run out of data, and therefore, it is desirable that the jitterbuffer 510 be spoofed with bit sequences. Preferably the bit sequencesare benign. In the described exemplary embodiment, the specific spoofingmethodology is dependent upon the common error mode protocol negotiatedby the error control logic of each network gateway.

In accordance with V.14 recommendations, the spoofing logic 516 checksfor character format and boundary (number of data bits, start bits andstop bits) within the jitter buffer 510. As specified in the V.14recommendation the spoofing logic 516 must account for stop bits omitteddue to asynchronous-to-synchronous conversion. Once the spoofing logic516 locates the character boundary, ones can be added to spoof the localmodem and keep the connection alive. The length of time a modem can bespoofed with ones depends only upon the application program driving thelocal modem.

In accordance with the V.42 recommendations, the spoofing logic 516checks for HDLC flag (HDLC frame boundary) within the jitter buffer 510.The basic HDLC structure consists of a number of frames, each of whichis subdivided into a number of fields. The HDLC frame structure providesfor frame labeling and error checking. When a new data transmission isinitiated, HDLC preamble in the form of synchronization sequences aretransmitted prior to the binary coded information. The HDLC preamble ismodulated bit streams of “01111110 (0x7e)”. The jitter buffer 510 shouldbe sufficiently large to guarantee that at least one complete HDLC frameis contained within the jitter buffer 510. The default length of an HDLCframe is 132 octets. The V.42 recommendations for error correction ofdata circuit terminating equipment (DCE) usingasynchronous-to-synchronous conversion does not specify a maximum lengthfor an HDLC frame. However, because the length of the frame affects theoverall memory required to implement the protocol, a information framelength larger than 260 octets is unlikely.

The spoofing logic 516 stores a threshold water mark (with a value setto be approximately equal to the maximum length of the HDLC frame).Spoofing is preferably activated when the number of packets stored inthe jitter buffer 510 drops to the predetermined threshold level. Whenspoofing is required, the spoofing logic 516 adds HDLC flags at theframe boundary as a complete frame is being reassembled and forwarded tothe transmit data pump. This continues until the number of data packetsin the jitter buffer 510 exceeds the threshold level.

4. Retrain and Rate Renegotiation

In the described exemplary embodiment, if data rates independentlynegotiated between the modems and their respective network gateways aredifferent, the rate negotiator will renegotiate the data rate byadopting the lowest of the two data rates. The call and answer modemswill then undergo retraining or rate renegotiation procedures by theirrespective network gateways to establish a new connection at therenegotiated data rate. In addition, rate synchronization may be lostduring a modem communication, requiring modem retraining and raterenegotiation, due to drift or change in the conditions of thecommunication channel. When a retrain occurs, an indication should beforwarded to the network gateway at the end of the packet based network.The network gateway receiving a retrain indication should initiateretrain with the connected modem to keep data flow in synchronismbetween the two connections. Rate synchronization procedures aspreviously described should be used to maintain data rate alignmentafter retrains.

Similarly, rate renegotiation causes both the calling and answer networkgateways and to perform rate renegotiation. However, rate signals or MP(CP) sequences should be exchanged per method two of the data ratealignment as previously discussed for a V.32bis or V.34 ratesynchronization whichever is appropriate.

5. Error Correcting Mode Synchronization

Error control (V.42) and data compression (V.42bis) modes should besynchronized at each end of the packet based network. In a first method,the call modem and the answer modem independently negotiate an errorcorrection mode with each other on their own, transparent to the networkgateways. This method is preferred for connections wherein the networkdelay plus jitter is relatively small, as characterized by an overallround trip delay of less than 700 msec.

Data compression mode is negotiated within V.42 so that the appropriatemode indication can be relayed when the calling and answer modems haveentered into V.42 mode.

An alternative method is to allow modems at both ends to freelynegotiate the error control mode with their respective network gateways.The network gateways must fully support all error correction modes whenusing this method. Also, this method cannot support the scenario whereone modem selects V.14 while the other modem selects a mode other thanV.14. For the case where V.14 is negotiated at both sides of the packetbased network, an 8-bit no parity format is assumed by each respectivenetwork gateway and the raw demodulated data bits are transported therebetween. With all other cases, each gateway shall extract de-framed(error corrected) data bits and forward them to its counterpart at theopposite end of the network. Flow control procedures within the errorcontrol protocol may be used to handle network delay. The advantage ofthis method over the first method is its ability to handle large networkdelays and also the scenario where the local connection rates at thenetwork gateways are different. However, packets transported over thenetwork in accordance with this method must be guaranteed to be errorfree. This may be achieved by establishing a connection between thenetwork gateways in accordance with the link access protocol connectionfor modems (LAPM)

6. Data Pump

Preferably, the data exchange includes a modem relay having a data pumpfor demodulating modem data signals from a modem for transmission on thepacket based network, and remodulating modem data signal packets fromthe packet based network for transmission to a local modem. Similarly,the data exchange also preferably includes a fax relay with a data pumpfor demodulating fax data signals from a fax for transmission on thepacket based network, and remodulating fax data signal packets from thepacket based network for transmission to a local fax device. Theutilization of a data pump in the fax and modem relays to demodulate andremodulate data signals for transmission across a packet based networkprovides considerable bandwidth savings. First, only the underlyingunmodulated data signals are transmitted across the packet basednetwork. Second, data transmission rates of digital signals across thepacket based network, typically 64 kbps is greater than the maximum rateavailable (typically 33,600 bps) for communication over a circuitswitched network.

Telephone line data pumps operating in accordance with ITU V seriesrecommendations for transmission rates of 2400 bps or more typicallyutilize quadrature amplitude modulation (QAM). A typical QAM data pumptransmitter 600 is shown schematically in FIG. 28. The transmitter inputis a serial binary data stream d_(n) arriving at a rate of R_(d) bps. Aserial to parallel converter 602 groups the input bits into J-bit binarywords. A constellation mapper 604 maps each J-bit binary word to achannel symbol from a 2^(J) element alphabet resulting in a channelsymbol rate of f_(s)=R_(d)/J baud. The alphabet consists of a pair ofreal numbers representing points in a two-dimensional space, called thesignal constellation. Customarily the signal constellation can bethought of as a complex plane so that the channel symbol sequence may berepresented as a sequence of complex numbers c_(n)=a_(n)+jb_(n).Typically the real part a_(n) is called the in-phase or I component andthe imaginary b_(n) is called the quadrature or Q component. A nonlinearencoder 605 may be used to expand the constellation points in order tocombat the negative effects of companding in accordance with ITU-T G.711standard. The I & Q components may be modulated by impulse modulators606 and 608 respectively and filtered by transmit shaping filters 610and 612 each with impulse response g_(T)(t). The outputs of the shapingfilters 610 and 612 are called in-phase 610(a) and quadrature 612(a)components of the continuous-time transmitted signal.

The shaping filters 610 and 612 are typically lowpass filtersapproximating the raised cosine or square root of raised cosineresponse, having a cutoff frequency on the order of at least aboutf_(s)/2. The outputs 610(a) and 612(a) of the lowpass filters 610 and612 respectively are lowpass signals with a frequency domain extendingdown to approximately zero hertz. A local oscillator 614 generatesquadrature carriers cos(ω_(c)t) 614(a) and sin(ω_(c)t) 614(b).Multipliers 616 and 618 multiply the filter outputs 610(a) and 612(a) byquadrature carriers cos(ω_(c)t) and sin(ω_(c)t) respectively toamplitude modulate the in-phase and quadrature signals up to thepassband of a bandpass channel. The modulated output signals 616(a) and618(a) are then subtracted in a difference operator 620 to form atransmit output signal 622. The carrier frequency should be greater thanthe shaping filter cutoff frequency to prevent spectral fold-over.

A data pump receiver 630 is shown schematically in FIG. 29. The datapump receiver 630 is generally configured to process a received signal630(a) distorted by the non-ideal frequency response of the channel andadditive noise in a transmit data pump (not shown) in the local modem.An analog to digital converter (A/D) 631 converts the received signal630(a) from an analog to a digital format. The A/D converter 631 samplesthe received signal 630(a) at a rate of f_(o)=1/T_(o)=n_(o)/T which isn_(o) times the symbol rate f_(s)=1/T and is at least twice the highestfrequency component of the received signal 630(a) to satisfy nyquistsampling theory.

An echo canceller 634 substantially removes the line echos on thereceived signal 630(a). Echo cancellation permits a modem to operate ina full duplex transmission mode on a two-line circuit, such as a PSTN.With echo cancellation, a modem can establish two high-speed channels inopposite directions. Through the use of digital-signal-processingcircuitry, the modem's receiver can use the shape of the modem'stransmitter signal to cancel out the effect of its own transmittedsignal by subtracting reference signal and the receive signal 630(a) ina difference operator 633.

Multiplier 636 scales the amplitude of echo cancelled signal 633(a). Apower estimator 637 estimates the power level of the gain adjustedsignal 636(a). Automatic gain control logic 638 compares the estimatedpower level to a set of predetermined thresholds and inputs a scalingfactor into the multiplier 636 that adjusts the amplitude of the echocanceled signal 633(a) to a level that is within the desired amplituderange. A carrier detector 642 processes the output of a digitalresampler 640 to determine when a data signal is actually present at theinput to receiver 630. Many of the receiver functions are preferably notinvoked until an input signal is detected.

A timing recovery system 644 synchronizes the transmit clock of theremote data pump transmitter (not shown) and the receiver clock. Thetiming recovery system 644 extracts timing information from the receivedsignal, and adjusts the digital resampler 640 to ensure that thefrequency and phase of the transmit clock and receiver clock aresynchronized. A phase splitting fractionally spaced equalizer (PSFSE)646 filters the received signal at the symbol rate. The PSFSE 646compensates for the amplitude response and envelope delay of the channelso as to minimize inter-symbol interference in the received signal. Thefrequency response of a typical channel is inexact so that an adaptivefilter is preferable. The PSFSE 646 is preferably an adaptive FIR filterthat operates on data signal samples spaced by T/n₀ and generatesdigital signal output samples spaced by the period T. In the describedexemplary embodiment n₀=3.

The PSFSE 646 outputs a complex signal which multiplier 650 multipliesby a locally generated carrier reference 652 to demodulate the PSFSEoutput to the baseband signal 650(a). The received signal 630(a) istypically encoded with a non-linear operation so as to reduce thequantization noise introduced by companding in accordance with ITU-TG.711. The baseband signal 650(a) is therefore processed by a non-lineardecoder 654 which reverses the non-linear encoding or warping. The gainof the baseband signal will typically vary upon transition from atraining phase to a data phase because modem manufacturers utilizedifferent methods to compute a scale factor. The problem that arises isthat digital modulation techniques such as quadrature amplitudemodulation (QAM) and pulse amplitude modulation (PAM) rely on precisegain (or scaling) in order to achieve satisfactory performance.Therefore, a scaling error compensator 656 adjusts the gain of thereceiver to compensate for variations in scaling. Further, a slicer 658then quantizes the scaled baseband symbols to the nearest idealconstellation points, which are the estimates of the symbols from theremote data pump transmitter (not shown). A decoder 659 converts theoutput of slicer 658 into a digital binary stream.

During data pump training, known transmitted training sequences aretransmitted by a data pump transmitter in accordance with the applicableITU-T standard. An ideal reference generator 660, generates a localreplica of the constellation point 660(a). During the training phase aswitch 661 is toggled to connect the output 660(a) of the idealreference generator 660 to a difference operator 662 that generates abaseband error signal 662(a) by subtracting the ideal constellationsequence 660(a) and the baseband equalizer output signal 650(a). Acarrier phase generator 664 uses the baseband error signal 662(a) andthe baseband equalizer output signal 650(a) to synchronize local carrierreference 666 with the carrier of the received signal 630(a) During thedata phase the switch 661 connects the output 658(a) of the slicer tothe input of difference operator 662 that generates a baseband errorsignal 662(a) in the data phase by subtracting the estimated symboloutput by the slicer 658 and the baseband equalizer output signal650(a). It will be appreciated by one of skill that the describedreceiver is one of several approaches. Alternate approaches inaccordance with ITU-T recommendations may be readily substituted for thedescribed data pump. Accordingly, the described exemplary embodiment ofthe data pump is by way of example only and not by way of limitation.

a. Timing Recovery System

Timing recovery refers to the process in a synchronous communicationsystem whereby timing information is extracted from the data beingreceived. In the context of a modem connection in accordance with anexemplary embodiment of the present invention, each modem is coupled toa signal processing system, which for the purposes of explanation isoperating in a network gateway, either directly or through a PSTN line.In operation, each modem establishes a modem connection with itsrespective network gateway, at which point, the modems begin relayingdata signals across a packet based network. The problem that arises isthat the clock frequencies of the modems are not identical to the clockfrequencies of the data pumps operating in their respective networkgateways. By design, the data pump receiver in the network gatewayshould sample a received signal of symbols in synchronism with thetransmitter clock of the modem connected locally to that gateway inorder to properly demodulate the transmitted signal.

A timing recovery system can be used for this purpose. Although thetiming recovery system is described in the context of a data pump withina signal processing system with the packet data modem exchange invoked,those skilled in the art will appreciate that the timing recovery systemis likewise suitable for various other applications in various othertelephony and telecommunications applications, including fax data pumps.Accordingly, the described exemplary embodiment of the timing recoverysystem in a signal processing system is by way of example only and notby way of limitation.

A block diagram of a timing recovery system is shown in FIG. 30. In thedescribed exemplary embodiment, the digital resampler 640 resamples thegain adjusted signal 636(a) output by the AGC (see FIG. 29). A timingerror estimator 670 provides an indication of whether the local timingor clock of the data pump receiver is leading or lagging the timing orclock of the data pump transmitter in the local modem. As is known inthe art, the timing error estimator 670 may be implemented by a varietyof techniques including that proposed by Godard. The A/D converter 631of the data pump receiver (see FIG. 29) samples the received signal630(a) at a rate of fo which is an integer multiple of the symbol ratefs=1/T and is at least twice the highest frequency component of thereceived signal 630(a) to satisfy nyquist sampling theory. The samplesare applied to an upper bandpass filter 672 and a lower bandpass filter674. The upper bandpass filter 672 is tuned to the upper bandedgefrequency fu=fc+0.5 fs and the lower bandpass filter 674 is tuned to thelower bandedge frequency fl=fc−0.5 fs where fc is the carrier frequencyof the QAM signal. The bandwidth of the filters 672 and 674 should bereasonably narrow, preferably on the order of 100 Hz for a fs=2400 baudmodem. Conjugate logic 676 takes the complex conjugate of complex outputof the lower bandpass filter. Multiplier 678 multiplies the complexoutput of the upper bandpass filter 672(a) by the complex conjugate ofthe lower bandpass filter to form a cross-correlation between the outputof the two filters (672 and 674). The real part of the correlated symbolis discarded by processing logic 680, and a sampler 681 samples theimaginary part of the resulting cross-correlation at the symbol rate toprovide an indication of whether the timing phase error is leading orlagging.

In operation, a transmitted signal from a remote data pump transmitter(not shown) g(t) is made to correspond to each data character. Thesignal element has a bandwidth approximately equal to the signaling ratefs. The modulation used to transmit this signal element consists ofmultiplying the signal by a sinusoidal carrier of frequency fc whichcauses the spectrum to be translated to a band around frequency fc.Thus, the corresponding spectrum is bounded by frequencies f1=fc−0.5 fsand f2=fc+0.5 fs, which are known as the bandedge frequencies. Referencefor more detailed information may be made to “Principles of DataCommunication” by R. W. Lucky, J. Salz and E. J. Weldon, Jr.,McGraw-Hill Book Company, pages 50-51.

In practice it has been found that additional filtering is required toreduce symbol clock jitter, particularly when the signal constellationcontains many points. Conventionally a loop filter 682 filters thetiming recovery signal to reduce the symbol clock jitter. Traditionallythe loop filter 682 is a second order infinite impulse response (IIR)type filter, whereby the second order portion tracks the offset in clockfrequency and the first order portion tracks the offset in phase. Theoutput of the loop filter drives clock phase adjuster 684. The clockphase adjuster controls the digital sampling rate of digital resampler640 so as to sample the received symbols in synchronism with thetransmitter clock of the modern connected locally to that gateway.Typically, the clock phase adjuster 684 utilizes a poly-phaseinterpolation algorithm to digitally adjust the timing phase. The timingrecovery system may be implemented in either analog or digital form.Although digital implementations are more prevalent in current modemdesign an analog embodiment may be realized by replacing the clock phaseadjuster with a VCO.

The loop filter 682 is typically implemented as shown in FIG. 31. Thefirst order portion of the filter controls the adjustments made to thephase of the clock (not shown) A multiplier 688 applies a first orderadjustment constant α to advance or retard the clock phase adjustment.Typically the constant α is empirically derived via computer simulationor a series of simple experiments with a telephone network simulator.Generally α is dependent upon the gain and the bandwidth of the upperand lower filters in the timing error estimator, and is generallyoptimized to reduce symbol clock jitter and control the speed at whichthe phase is adjusted. The structure of the loop filter 682 may includea second order component 690 that estimates the offset in clockfrequency. The second order portion utilizes an accumulator 692 in afeedback loop to accumulate the timing error estimates. A multiplier 694is used to scale the accumulated timing error estimate by a constant β.Typically, the constant β is empirically derived based on the amount offeedback that will cause the system to remain stable. Summer 695 sumsthe scaled accumulated frequency adjustment 694(a) with the scaled phaseadjustment 688(a). A disadvantage of conventional designs which includea second order component 690 in the loop filter 682 is that such secondorder components 690 are prone to instability with large constellationmodulations under certain channel conditions.

An alternative digital implementation eliminates the loop filter.Referring to FIG. 32 a hard limiter 695 and a random walk filter 696 arecoupled to the output of the timing error estimator 680 to reduce timingjitter. The hard limiter 695 provides a simple automatic gain controlaction that keeps the loop gain constant independent of the amplitudelevel of the input signal. The hard limiter 695 assures that timingadjustments are proportional to the timing of the data pump transmitterof the local modem and not the amplitude of the received signal. Therandom walk filter 696 reduces the timing jitter induced into the systemas disclosed in “Communication System Design Using DSP Algorithms”, S.Tretter, p. 132, Plenum Press, NY., 1995, the contents of which ishereby incorporated by reference as through set forth in full herein.The random walk filter 696 acts as an accumulator, summing a randomnumber of adjustments over time. The random walk filter 696 is resetwhen the accumulated value exceeds a positive or negative threshold.Typically, the sampling phase is not adjusted so long as the accumulatoroutput remains between the thresholds, thereby substantially reducing oreliminating incremental positive adjustments followed by negativeadjustments that otherwise tend to not accumulate.

Referring to FIG. 33 in an exemplary embodiment of the presentinvention, the multiplier 688 applies the first order adjustmentconstant α to the output of the random walk filter to advance or retardthe estimated clock phase adjustment. In addition, a timing frequencyoffset compensator 697 is coupled to the timing recovery system viaswitches 698 and 699 to preferably provide a fixed dc component tocompensate for clock frequency offset present in the received signal.The exemplary timing frequency offset compensator preferably operates inphases. A frequency offset estimator 700 computes the total frequencyoffset to apply during an estimation phase and incremental logic 701,incrementally applies the offset estimate in linear steps during theapplication phase. Switch control logic 702 controls the toggling ofswitches 698 and 699 during the estimation and application phases ofcompensation adjustment. Unlike the second order component 690 of theconventional timing recovery loop filter disclosed in FIG. 31, thedescribed exemplary timing frequency offset compensator 697 is an openloop design such that the second order compensation is fixed duringsteady state. Therefore, switches 698 and 699 work in oppositecooperation when the timing compensation is being estimated and when itis being applied.

During the estimation phase, switch control logic 702 closes switch 698thereby coupling the timing frequency offset compensator 697 to theoutput of the random walk filter 696, and opens switch 699 so thattiming adjustments are not applied during the estimation phase. Thefrequency offset estimator 700 computes the timing frequency offsetduring the estimation phase over K symbols in accordance with the blockdiagram shown in FIG. 34. An accumulator 703 accumulates the frequencyoffset estimates over K symbols. A multiplier 704 is used to average theaccumulated offset estimate by applying a constant γ/K. Typically theconstant γ is empirically derived and is preferably in the range ofabout 0.5-2. Preferably K is as large as possible to improve theaccuracy of the average. K is typically greater than about 500 symbolsand less than the recommended training sequence length for the modem inquestion. In the exemplary embodiment the first order adjustmentconstant α is preferably in the range of about 100-300 part per million(ppm). The timing frequency offset is preferably estimated during thetiming training phase (timing tone) and equalizer training phase basedon the accumulated adjustments made to the clock phase adjuster 684 overa period of time.

During steady state operation when the timing adjustments are applied,switch control logic 702 opens switch 698 decoupling the timingfrequency offset compensator 697 from the output of the random walkfilter, and closes switch 699 so that timing adjustments are applied bysummer 705. After K symbols of a symbol period have elapsed and thefrequency offset compensation is computed, the incremental logic 701preferably applies the timing frequency offset estimate in incrementallinear steps over a period of time to avoid large sudden adjustmentswhich may throw the feedback loop out of lock. This is the transientphase. The length of time over which the frequency offset compensationis incrementally applied is empirically derived, and is preferably inthe range of about 200-800 symbols. After the incremental logic 701 hasincrementally applied the total timing frequency offset estimatecomputed during the estimate phase, a steady state phase begins wherethe compensation is fixed. Relative to conventional second order loopfilters, the described exemplary embodiment provides improved stabilityand robustness.

b. Multipass Training

Data pump training refers to the process by which training sequences areutilized to train various adaptive elements within a data pump receiver.During data pump training, known transmitted training sequences aretransmitted by a data pump transmitter in accordance with the applicableITU-T standard. In the context of a modem connection in accordance withan exemplary embodiment of the present invention, the modems (see FIG.24) are coupled to a signal processing system, which for the purposes ofexplanation is operating in a network gateway, either directly orthrough a PSTN line. In operation, the receive data pump operating ineach network gateway of the described exemplary embodiment utilizesPSFSE architecture. The PSFSE architecture has numerous advantages overother architectures when receiving QAM signals. However, the PSFSEarchitecture has a slow convergence rate when employing the least meansquare (LMS) stochastic gradient algorithm. This slow convergence ratetypically prevents the use of PSFSE architecture in modems that employrelatively short training sequences in accordance with common standardssuch as V.29. Because of the slow convergence rate, the describedexemplary embodiment re-processes blocks of training samples multipletimes (multi-pass training).

Although the method of performing multi-pass training is described inthe context of a signal processing system with the packet data exchangeinvoked, those skilled in the art will appreciate that multi-passtraining is likewise suitable for various other telephony andtelecommunications applications. Accordingly, the described exemplarymethod for multi-pass training in a signal processing system is by wayof example only and not by way of limitation.

In an exemplary embodiment the data pump receiver operating in thenetwork gateway stores the received QAM samples of the modem's trainingsequence in a buffer until N symbols have been received. The PSFSE isthen adapted sequentially over these N symbols using a LMS algorithm toprovide a coarse convergence of the PSFSE. The coarsely converged PSFSE(i.e. with updated values for the equalizer taps) returns to the startof the same block of training samples and adapts a second time. Thisprocess is repeated M times over each block of training samples. Each ofthe M iterations provides a more precise or finer convergence until thePSFSE is completely converged.

c. Scaling Error Compensator

Scaling error compensation refers to the process by which the gain of adata pump receiver (fax or modem) is adjusted to compensate forvariations in transmission channel conditions. In the context of a modemconnection in accordance with an exemplary embodiment of the presentinvention, each modem is coupled to a signal processing system, whichfor the purposes of explanation is operating in a network gateway,either directly or through a PSTN line. In operation, each modemcommunicates with its respective network gateway using digitalmodulation techniques. The problem that arises is that digitalmodulation techniques such as QAM and pulse amplitude modulation (PAM)rely on precise gain (or scaling) in order to achieve satisfactoryperformance. In addition, transmission in accordance with the V.34recommendations typically includes a training phase and a data phasewhereby a much smaller constellation size is used during the trainingphase relative to that used in the data phase. The V.34 recommendation,requires scaling to be applied when switching from the smallerconstellation during the training phase into the larger constellationduring the data phase.

The scaling factor can be precisely computed by theoretical analysis,however, different manufacturers of V.34 systems (modems) tend to useslightly different scaling factors. Scaling factor variation (or error)from the predicted value may degrade performance until the PSFSEcompensates for the variation in scaling factor. Variation in gain dueto transmission channel conditions is compensated by an initial gainestimation algorithm (typically consisting of a simple signal powermeasurement during a particular signaling phase) and an adaptiveequalizer during the training phase. However, since a PSFSE ispreferably configured to adapt very slowly during the data phase, theremay be a significant number of data bits received in error before thePSFSE has sufficient time to adapt to the scaling error.

It is, therefore, desirable to quickly reduce the scaling error andhence minimize the number of potential erred bits. A scaling factorcompensator can be used for this purpose. Although the scaling factorcompensator is described in the context of a signal processing systemwith the packet data modem exchange invoked, those skilled in the artwill appreciate that the preferred scaling factor compensator islikewise suitable for various other telephony and telecommunicationsapplications. Accordingly, the described exemplary embodiment of thescaling factor compensator in a signal processing system is by way ofexample only and not by way of limitation.

FIG. 35 shows a block diagram of an exemplary embodiment of the scalingerror compensator in a data pump receiver 630 (see FIG. 29). In anexemplary embodiment, scaling error compensator 708 computes the gainadjustment of the data pump receiver. Multiplier 710 adjusts a nominalscaling factor 712 (the scaling error computed by the data pumpmanufacturer) by the gain adjustment as computed by the scaling errorcompensator 708. The combined scale factor 710(a) is applied to theincoming symbols by multiplier 714. A slicer 716 quantizes the scaledbaseband symbols to the nearest ideal constellation points, which arethe estimates of the symbols from the remote data pump transmitter.

The scaling error compensator 708 preferably includes a divider 718which estimates the gain adjustment of the data pump receiver bydividing the expected magnitude of the received symbol 716(a) by theactual magnitude of the received symbol 716(b). In the describedexemplary embodiment the magnitude is defined as the sum of squaresbetween real and imaginary parts of the complex symbol. The expectedmagnitude of the received symbol is the output 716(a) of the slicer 716(i.e. the symbol quantized to the nearest ideal constellation point)whereas the magnitude of the actual received symbol is the input 716(b)to the slicer 716. In the case where a Viterbi decoder performs theerror-correction of the received, noise-disturbed signal (as for V.34),the output of the slicer may be replaced by the first level decision ofthe Viterbi decoder.

The statistical nature of noise is such that large spikes in theamplitude of the received signal will occasionally occur. A large spikein the amplitude of the received signal may result in an erroneouslylarge estimate of the gain adjustment of the data pump receiver.Typically, scaling is applied in a one to one ratio with the estimate ofthe gain adjustment, so that large scaling factors may be erroneouslyapplied when large amplitude noise spikes are received. To minimize theimpact of large amplitude spikes and improve the accuracy of the system,the described exemplary scaling error compensator 708 further includes anon-linear filter in the form of a hard-limiter 720 which is applied toeach estimate 718(a). The hard limiter 720 limits the maximum adjustmentof the scaling value. The hard limiter 720 provides a simple automaticcontrol action that keeps the loop gain constant independent of theamplitude of the input signal so as to minimize the negative effects oflarge amplitude noise spikes. In addition, averaging logic 722 computesthe average gain adjustment estimate over a number (N) of symbols in thedata phase prior to adjusting the nominal scale factor 710. As will beappreciated by those of skill in the art, other non-linear filteringalgorithms may also be used in place of the hard-limiter.

Alternatively, the accuracy of the scaling error compensation may befurther improved by estimating the averaged scaling adjustment twice andapplying that estimate in two steps. A large hard limit value (typically1+/−0.25) is used to compute the first average scaling adjustment. Theinitial prediction provides an estimate of the average value of theamplitude of the received symbols. The unpredictable nature of theamplitude of the received signal requires the use of a large initialhard limit value to ensure that the true scaling error is included inthe initial estimate of the average scaling adjustment. The estimate ofthe average value of the amplitude of the received symbols is used tocalibrate the limits of the scaling adjustment. The average scalingadjustment is then estimated a second time using a lower hard limitvalue and then applied to the nominal scale factor 712 by multiplier710.

In most modem specifications, such as the V.34 standards, there is adefined signaling period (B1 for V.34) after transition into data phasewhere the data phase constellation is transmitted with signalinginformation to flush the receiver pipeline (i.e. Viterbi decoder etc.)prior to the transmission of actual data. In an exemplary embodimentthis signaling period may be used to make the scaling adjustment suchthat any scaling error is compensated for prior to actual transfer ofdata.

d. Non-Linear Decoder

In the context of a modem connection in accordance with an exemplaryembodiment of the present invention, each modem is coupled to a signalprocessing system, which for the purposes of explanation is operating ina network gateway, either directly or through a PSTN line. In operation,each modem communicates with its respective network gateway usingdigital modulation techniques. The international telecommunicationsunion (ITU) has promulgated standards for the encoding and decoding ofdigital data in ITU-T Recommendation G.711 (ref. G.711) which isincorporated herein by reference as if set forth in full. The encodingstandard specifies that a nonlinear operation (companding) be performedon the analog data signal prior to quantization into seven bits plus asign bit. The companding operation is a monatomic invertable functionwhich reduces the higher signal levels. At the decoder, the inverseoperation (expanding) is done prior to analog reconstruction. Thecompanding/expanding operation quantizes the higher signal values morecoarsely. The companding/expanding operation, is suitable for thetransmission of voice signals but introduces quantization noise on datamodem signals. The quantization error (noise) is greater for the outersignal levels than the inner signal levels.

The ITU-T Recommendation V.34 describes a mechanism whereby (ref. V.34)the uniform signal is first expanded (ref. BETTS) to space the outerpoints farther apart than the inner points before G.711 encoding andtransmission over the PCM link. At the receiver, the inverse operationis applied after G.711 decoding. The V.34 recommended expansion/inverseoperation yields a more uniform signal to noise ratio over the signalamplitude. However, the inverse operation specified in the ITU-TRecommendation V.34 requires a complex receiver calculation. Thecalculation is computationally intensive, typically requiring numerousmachine cycles to implement.

It is, therefore, desirable to reduce the number of machine cyclesrequired to compute the inverse to within an acceptable error level. Asimplified nonlinear decoder can be used for this purpose. Although thenonlinear decoder is described in the context of a signal processingsystem with the packet data modem exchange invoked, those skilled in theart will appreciate that the nonlinear decoder is likewise suitable forvarious other telephony and telecommunications application. Accordingly,the described exemplary embodiment of the nonlinear decoder in a signalprocessing system is by way of example only and not by way oflimitation.

Conventionally, iteration algorithms have been used to compute theinverse of the G.711 nonlinear warping function. Typically, iterationalgorithms generate an initial estimate of the input to the nonlinearfunction and then compute the output. The iteration algorithm comparesthe output to a reference value and adjusts the input to the nonlinearfunction. A commonly used adjustment is the successive approximationwherein the difference between the output and the reference function isadded to the input. However, when using the successive approximationtechnique, up to ten iterations may be required to adjust the estimatedinput of the nonlinear warping function to an acceptable error level, sothat the nonlinear warping function must be evaluated ten times. Thesuccessive approximation technique is computationally intensive,requiring significant machine cycles to converge to an acceptableapproximation of the inverse of the nonlinear warping function.Alternatively, a more complex warping function is a linear NewtonRhapson iteration. Typically the Newton Rhapson algorithm requires threeevaluations to converge to an acceptable error level. However, the innercomputations for the Newton Rhapson algorithm are more complex thanthose required for the successive approximation technique. The NewtonRhapson algorithm utilizes a computationally intensive iteration loopwherein the derivative of the nonlinear warping function is computed foreach approximation iteration, so that significant machine cycles arerequired to conventionally execute the Newton Rhapson algorithm.

An exemplary embodiment of the present invention modifies the successiveapproximation iteration. A presently preferred algorithm computes anapproximation to the derivative of the nonlinear warping function oncebefore the iteration loop is executed and uses the approximation as ascale factor during the successive approximation iterations. Thedescribed exemplary embodiment converges to the same acceptable errorlevel as the more complex conventional Newton-Rhapson algorithm in fouriterations. The described exemplary embodiment further improves thecomputational efficiency by utilizing a simplified approximation of thederivative of the nonlinear warping function.

In operation, development of the described exemplary embodiment proceedsas follows with a warping function defined as:

${w(v)} = {\frac{\Theta (v)}{6} + \frac{{\Theta (v)}^{2}}{120}}$

the V.34 nonlinear decoder can be written as

Y=X(1+w(∥X∥ ²))

taking the square of the magnitude of both sides yields,

Y ² =|X| ²(1+w(∥X∥ ²))²

The encoder notation can then be simplified with the followingsubstitutions

Y _(r) =∥Y∥ ² , X _(r) =∥X∥ ²

and write the V.34 nonlinear encoder equation in the cannonical formG(x)=0.

X _(r)(1+w(x _(r)))² −Y _(r)=0

The Newton-Rhapson iteration is a numerical method to determine X thatresults in an iteration of the form:

$X_{n + 1} = {X_{n} - \frac{G({Xn})}{G^{\prime}({Xn})}}$

where G′ is the derivative and the substitution iteration results whenG′ is set equal to one.

The computational complexity of the Newton-Rhapson algorithm is thuspaced by the derivation of the derivative G′, which conventionally isrelated to X_(r) so that the mathematical instructions saved byperforming fewer iterations are offset by the instructions required tocalculate the derivative and perform the divide. Therefore, it would bedesirable to approximate the derivative G′ with a term that is thefunction of the input Y_(r) so that G(x) is a monotonic function andG′(x) can be expressed in terms of G(x). Advantageously, if the steps inthe iteration are small, then G′(x) will not vary greatly and can beheld constant over the iteration. A series of simple experiments yieldsthe following approximation of G′(x) where α is an experimentallyderived scaling factor.

$G^{\prime} = \frac{1 + {Yr}}{\alpha}$

The approximation for G′ converges to an acceptable error level in aminimum number of steps, typically one more iteration than the fulllinear Newton-Rhapson algorithm. A single divide before the iterationloop computes the quantity

$\frac{1}{G^{\prime}} = \frac{\alpha}{1 + {Yr}}$

The error term is multiplied by 1/G′ in the successive iteration loop.It will be appreciated by one of skill in the art that furtherimprovements in the speed of convergence are possible with the“Generalized Newton-Rhapson” class of algorithms. However, the innerloop computations for this class of algorithm are quite complex.

Advantageously, the described exemplary embodiment does not expand thepolynomial because the numeric quantization on a store in a sixteen bitmachine may be quite significant for the higher order polynomial terms.The described exemplary embodiment organizes the inner loop computationsto minimize the effects of truncation and the number of instructionsrequired for execution. Typically the inner loop requires eighteeninstructions and four iterations to converge to within two bits of theactual value which is within the computational roundoff noise of asixteen bit machine.

D. Human Voice Detector

In a preferred embodiment of the present invention, a signal processingsystem is employed to interface telephony devices with packet basednetworks. Telephony devices include, by way of example, analog anddigital phones, ethernet phones, Internet Protocol phones, fax machines,data modems, cable voice modems, interactive voice response systems,PBXs, key systems, and any other conventional telephony devices known inthe art. In the described exemplary embodiment the packet voice exchangeis common to both the voice mode and the voiceband data mode. In thevoiceband data mode, the network VHD invokes the packet voice exchangefor transparently exchanging data without modification (other thanpacketization) between the telephony device or circuit switched networkand the packet based network. This is typically used for the exchange offax and modem data when bandwidth concerns are minimal as an alternativeto demodulation and remodulation.

During the voiceband data mode, the human voice detector service is alsoinvoked by the resource manager. The human voice detector monitors thesignal from the near end telephony device for voice. The describedexemplary human voice detector estimates pitch period of an incomingtelephony signal and compares the pitch period of said telephony signalto a plurality of thresholds to identify active voice samples. Thisapproach is substantially independent of the amplitude of the spokenutterance, so that whispered or shouted utterance may be accuratelyidentified as active voice samples. In the event that voice is detectedby the human voice detector, an event is forwarded to the resourcemanager which, in turn, causes the resource manager to terminate thehuman voice detector service and invoke the appropriate services for thevoice mode (i.e., the call discriminator, the packet tone exchange, andthe packet voice exchange).

Although a preferred embodiment is described in the context of a signalprocessing system for telephone communications across the packet basednetwork, it will be appreciated by those skilled in the art that thevoice detector is likewise suitable for various other telephony andtelecommunications application. Accordingly, the described exemplaryembodiment of the voice detector in a signal processing system is by wayof example only and not by way of limitation.

There are a variety of encoding methods known for encoding voice. Mostfrequently, voice is modeled on a short-time basis as the response of alinear system excited by a periodic impulse train for voiced sounds orrandom noise for the unvoiced sounds. Conventional human voice detectorstypically monitor the power level of the incoming signal to make avoice/machine decision. Typically, if the power level of the incomingsignal is above a predetermined threshold, the sequence is typicallydeclared voice. The performance of such conventional voice detectors maybe degraded by the environment, in that a very soft spoken whisperedutterance will have a very different power level from a loud shout. Ifthe threshold is set at too low a level, noise will be declared voice,whereas if the threshold is set at too high a level a soft spoken voicesegment will be incorrectly marked as inactive.

Alternatively, voice may generally be classified as voiced if afundamental frequency is imported to the air stream by the vocal cordsof the speaker. In such case, the frequency of a voice segment istypically highly periodic at around the pitch frequency. Thedetermination as to whether a voice segment is voiced or unvoiced, andthe estimation of the fundamental frequency can be obtained in a varietyof ways known in the art such as pitch detection algorithms. In thedescribed exemplary embodiment, the human voice detector calculates anautocorrelation function for the incoming signal. An autocorrelationfunction for a voice segment demonstrates local peaks with a periodicityin proportion to the pitch period. The human voice detector serviceutilizes this feature in conjunction with power measurements todistinguish voice signals from modem signals. It will be appreciatedthat other pitch detection algorithms known in the art can be used aswell.

Referring to FIG. 36, in the described exemplary embodiment, a powerestimator 730 estimates the power level of the incoming signal.Autocorrelation logic 732 computes an autocorrelation function for aninput signal to assist in the voice/machine decision. Autocorrelation,as is known in the art, involves correlating a signal with itself. Acorrelation function shows how similar two signals are, and how long thesignals remain similar when one is shifted with respect to the other.Periodic signals go in and out of phase as one is shifted with respectto the other, so that a periodic signal will show strong correlation atshifts where the peaks coincide. Thus, the autocorrelation of a periodicsignal is itself a periodic signal, with a period equal to the period ofthe original signal.

The autocorrelation calculation computes the autocorrelation functionover an interval of 360 samples with the following approach:

${R\lbrack k\rbrack} = {\sum\limits_{n = 0}^{N - k - 1}{{x\lbrack n\rbrack}{x\left\lbrack {n + k} \right\rbrack}}}$

where N=360, k=0,1,2 . . . 179.

A pitch tracker 734 estimates the period of the computed autocorrelationfunction. Framed based decision logic 736 analyzes the estimated powerlevel 730 a, the autocorrelation function 732 a and the periodicity 734a of the incoming signal to execute a frame based voice/machine decisionaccording to a variety of factors. For example, the energy of the inputsignal should be above a predetermined threshold level, preferably inthe range of about −45 to −55 dBm, before the frame based decision logic736 declares the signal to be voice. In addition, the typical pitchperiod of a voice segment should be in the range of about 60-400 Hz, sothat the autocorrelation function should preferably be periodic with aperiod in the range of about 60-400 Hz before the frame based decisionlogic 736 declares a signal as active or containing voice.

The amplitude of the autocorrelation function is a maximum for R[0],i.e. when the signal is not shifted relative to itself. Also, for aperiodic voice signal, the amplitude of the autocorrelation functionwith a one period shift (i.e. R[pitch period]) should preferably be inthe range of about 0.25-0.40 of the amplitude of the autocorrelationfunction with no shift (i.e. R[0]). Similarly, modem signaling mayinvolve certain DTMF or MF tones, in this case the signals are highlycorrelated, so that if the largest peak in the amplitude of theautocorrelation function after R[0] is relatively close in magnitude toR[0], preferably in the range of about 0.75-0.90 R[0], the frame baseddecision logic 736 declares the sequence as inactive or not containingvoice.

Once a decision is made on the current frame as to voice or machine,final decision logic 738 compares the current frame decision with thetwo adjacent frame decisions. This check is known as backtracking. If adecision conflicts with both adjacent decisions it is flipped, i.e.voice decision turned to machine and vice versa.

Although a preferred embodiment of the present invention has beendescribed, it should not be construed to limit the scope of the appendedclaims. For example, the present invention can be implemented by both asoftware embodiment or a hardware embodiment. Those skilled in the artwill understand that various modifications may be made to the describedembodiment. Moreover, to those skilled in the various arts, theinvention itself herein will suggest solutions to other tasks andadaptations for other applications. It is therefore desired that thepresent embodiments be considered in all respects as illustrative andnot restrictive, reference being made to the appended claims rather thanthe foregoing description to indicate the scope of the invention.

1-58. (canceled)
 59. A system for cancelling a far end echo from a nearend signal, the system comprising: at least one processor forcommunicatively coupling to the near-end signal, the at least oneprocessor operable to, at least: estimate an energy level of the far endecho; cancel the echo from the near end signal, if the estimated energylevel of the far end echo is above an audible level; bypass thecancelling, if the estimated energy level of the far end echo is belowthe audible level; and control convergence of an adaptive filterresponsive to the estimated energy level of the far end echo, whereinestimating the energy level of the far end echo comprises estimating apower level of the far end signal, estimating an echo return lossbetween the far end signal and the near end signal, and estimating apower level for noise on the near end signal without the echo, andwherein the echo is canceled from the near end signal when the powerlevel of the far end signal minus the echo return loss is greater thanboth a threshold of hearing and the power level for the noise minus anamount in the range of 8-12 dB.
 60. The system of claim 59, whereinestimating the energy level of the far end echo comprises estimating apower level of the far end signal, and estimating an echo return lossbetween the far end signal and the near end signal, and wherein the echois cancelled from the near end signal if the estimated power level ofthe far end signal minus the echo return loss is greater than athreshold.
 61. The system of claim 59, wherein estimating the energylevel of the far end echo comprises estimating a power level of the farend signal, estimating an echo return loss between the far end signaland the near end signal, and estimating a power level of the near endsignal, wherein the selection of whether to cancel the echo from thenear end signal is based on the estimated power levels and the estimatedecho return loss.
 62. The system of claim 59, wherein the echocancellation comprises adaptively filtering the far end signal andsubtracting the filtered far end signal from the near end signal. 63.The method of claim 62, wherein estimating the energy level of the farend echo comprises estimating an echo return loss between the far endsignal and the near end signal, and estimating an echo return lossenhancement between the near end signal and the near end signal withoutthe echo, and wherein filter adaptation is a function of at least one ofthe echo return loss and echo return loss enhancement.
 64. The system ofclaim 63, wherein the filter adaptation comprises using an adaptationstep size of one-fourth when the echo return loss enhancement is in therange of 0-9 dBm.
 65. The system of claim 63, wherein the filteradaptation comprises using an adaptation step size of 1/32 when acombination of the estimated echo return loss and the echo return lossenhancement is greater than 33-36 dB.
 66. The system of claim 63,wherein the filter adaptation comprises using an adaptation step size of1/16 when a combination of the estimated echo return loss and the echoreturn loss enhancement is in the range of 23-33 dB.
 67. The system ofclaim 62, wherein the at least one processor is operable to, at least:detect information in the near end signal, wherein the filter adaptationcomprises limiting the filter adaptation when the information isdetected and the filter adaptation is converged.
 68. The system of claim67, wherein the filter adaptation comprises using an adaptation stepsize of 1/32 when the information is detected and the filter adaptationis not converged.
 69. The system of claim 67, wherein the limiting ofthe filter adaptation comprises disabling the filter adaptation.
 70. Thesystem of claim 62, wherein the filter adaptation is limited when thefilter adaptation has been active for a period longer than one secondfrom an off hook transition of a telephony device connected between thefar end signal and the near end signal.
 71. The system of claim 62,wherein the filter adaptation is limited when the filter adaptation hasbeen active for a period longer than one second after filter adaptationinitialization.
 72. The system of claim 62, wherein estimating theenergy level of the far end echo comprises estimating a power level ofthe far end signal, and estimating a power level for noise on the nearend signal without the echo, and wherein the filter adaptation comprisesusing an adaptation step size of ¼ when the estimated power level of thefar end signal exceeds the estimated power level of the noise by atleast 24 dB.
 73. The system of claim 62, wherein estimating the energylevel of the far end echo comprises estimating a power level of the farend signal, and estimating a power level for noise on the near endsignal without the echo, and wherein the filter adaptation comprisesusing an adaptation step size of ⅛ when the estimated power level of thefar end signal exceeds the estimated power level of the noise by atleast 18 dB.
 74. The system of claim 62, wherein estimating the energylevel of the far end echo comprises estimating a power level of the farend signal, and estimating a power level for noise on the near endsignal without the echo, and wherein the filter adaptation comprisesusing an adaptation step size of 1/16 when the estimated power level ofthe far end signal exceeds the estimated power level of the noise by atleast 9 dB.
 75. The system of claim 62, wherein the at least oneprocessor is operable to, at least: selectively limit filter adaptation,the selection of whether to limit the filter adaptation being based onthe estimated characteristic.
 76. The system of claim 59, wherein the atleast one processor is operable to, at least: detect information in thefar end signal, detecting information in the near end signal, andprocessing the near end signal when information is detected in the farend signal and not in the near end signal.
 77. The system of claim 76,wherein the near end is processed by attenuation.
 78. The system ofclaim 76, wherein the processing of the near end signal is non-linear.79. The system of claim 59, wherein estimating the energy level of thefar end echo comprises estimating a power level of the far end signal,estimating a power level of the near end signal, estimating a powerlevel of a near end signal without the echo, estimating a power level ofnoise on the far end signal, and selectively non linear processing thenear end signal, the selection as to whether to non linear process thenear end signal being based on the estimated power levels.
 80. Thesystem of claim 79, wherein the at least one processor is operable to,at least: set a first decision variable as a function of the estimatedpower level of the far end signal, setting a second decision variable asa function of the power level of the near end signal without the echo,setting a third decision variable as a function of the estimated powerlevel of the far end signal and the near end signal without the echo,wherein the near end signal is non linear processed when at least one ofthe three decision variables meet a respective criteria.
 81. The systemof claim 80, wherein the first decision variable is set when theestimated power level of the far end signal is at least 6 dB greaterthan the estimated power level of the noise on the far end signal, andthe estimated power level of the far end signal minus an estimated echoreturn loss between the far end signal and the near end signal is atleast 6 dB greater than the estimated power level of the near endsignal.
 82. The system of claim 80, wherein the second decision variableis set when the estimated power level of the near end signal without theecho is at least 9 dB less than the estimated power level of the nearend signal.
 83. The system of claim 80, wherein the third decisionvariable is set when the estimated power level of the far end signalminus the estimated power level of the near end signal without the echois greater than a threshold power level.
 84. A system for conditioning acomposite signal, the composite signal being formed by introducing atleast a portion of a first signal into a second signal, the systemcomprising: at least one processor for receiving the composite signal,the at least one processor operable to, at least: estimate a signalcharacteristic of the first signal; estimate a signal characteristic ofthe composite signal; recover the second signal from the compositesignal, if the estimated signal characteristic of the first signal andof the composite signal are above a predetermined level, whereinrecovering the second signal comprises filtering the first signal usingan adaptive filter, and subtracting the filtered first signal from thecomposite signal to recover the second signal; selectively enablerecovery of the second signal from the composite signal, if theestimated signal characteristic of the first signal and the compositesignal are below the predetermined level, wherein recovery is enabledwhen the estimated maximum power level of the first signal minus theestimated return loss is greater than both a threshold of hearing andthe estimated power level of the noise of the recovered second signalminus 8 dB; control convergence of an adaptive filter responsive to theestimated signal characteristics; estimate a maximum power level of thefirst signal; estimate a noise power level for the recovered secondsignal; and estimate a return loss between the first signal and thecomposite signal.
 85. The system of claim 84, wherein the at least oneprocessor is operable to, at least: estimate a maximum power level andan average power level of the first signal, and estimate a return lossbetween the first signal and the composite signal, wherein recovery ofthe second signal is enabled as a function of at least one of theestimated maximum power level, the estimated average power level, andthe estimated return loss.
 86. The system of claim 85, wherein the atleast one processor is operable to, at least: estimate an average powerlevel of the composite signal, wherein the system estimates the returnloss by dividing the estimated average power level of the first signalby the estimated average power level of the composite signal.
 87. Thesystem of claim 86, wherein recovery of the second signal is enabledwhen the estimated maximum power level of the first signal minus theestimated return loss is at least 8 dB greater than the estimated powerlevel of the composite signal.
 88. The system of claim 84, wherein theat least one processor is operable to, at least: suppress the recoveredsecond signal when information is detected in the first signal but notin the composite signal.
 89. The system of claim 88, wherein theinformation includes voice.
 90. The system of claim 88, whereinsuppression is performed in a non-linear manner.
 91. The system of claim90, wherein the filter adaptation is limited by disabling the adaptationof the adaptive filter.
 92. The system of claim 84, wherein the at leastone processor is operable to, at least: adjust the adaptation of theadaptive filter.
 93. The system of claim 92, wherein the at least oneprocessor limits the adaptation of the adaptive filter when recovery ofthe second signal is not enabled.
 94. The system of claim 92, whereinthe at least one processor is operable to, at least: estimate a returnloss between the first signal and the composite signal, and a returnloss enhancement between the composite signal and the recovered secondsignal, and adjust the adaptation of the adaptive filter as a functionof the estimated return loss and the estimated return loss enhancement.95. The signal conditioner of claim 94, wherein the at least oneprocessor is operable to, at least: estimate a maximum power level andan average power level of the first signal, estimate an average powerlevel of the composite signal, and estimate an average power level and anoise power level for the recovered second signal, wherein the returnloss and the return loss enhancement are estimated as a function of theestimated power levels.
 96. The system of claim 95, wherein the returnloss is estimated by dividing the average power level of the firstsignal by the average power level of the composite signal.
 97. Thesystem of claim 95, wherein the return loss enhancement is estimated bydividing the average power of the composite signal by the average powerof the recovered second signal.
 98. The system of claim 95, wherein theat least one processor causes the adaptive filter to have a filteradaptation step size of ¼ when the estimated average power level of thefirst signal is 24 dB greater than the estimated power level of thenoise of the recovered second signal.
 99. The system of claim 95,wherein the at least one processor causes the adaptive filter to have afilter adaptation step size of ⅛ when the estimated average power levelof the first signal is 18 dB greater than the estimated power level ofthe noise on the recovered second signal.
 100. The system of claim 95,wherein the at least one processor causes the adaptive filter to have afilter adaptation step size of 1/16 when the estimated average powerlevel of the first signal is 9 dB greater than the estimated power levelof the noise on the recovered second signal.
 101. The system of claim91, wherein the at least one processor causes the adaptive filter tohave an adaptation step size of 1/16 when a combination of the estimatedreturn loss and the estimated return loss enhancement is in the range ofabout 23-33 dB.
 102. The system of claim 91, wherein the at least oneprocessor limits adaptation of the adaptive filter when convergence ofinformation in the composite signal and the adaptive filter is detected.103. The system of claim 102, wherein the information includes voice.104. The system of claim 91, wherein the at least one processor limitsadaptation of the adaptive filter when the adaptive filter has beenactive for a period longer than one second after an off hook transitionof a telephony device coupled between the first signal and the compositesignal.
 105. The system of claim 91, wherein the at least one processorlimits the adaptation of the adaptive filter when the adaptive filterhas been active for a period longer than one second after the adaptivefilter is initialized.
 106. The system of claim 91, wherein the at leastone processor causes the adaptive filter to have an adaptation step sizeof 1/32 when information is detected in the composite signal and theadaptive filter is not converged.
 107. The system of claim 91, whereinthe at least one processor causes the adaptive filter to have anadaptation step size of one-fourth when the estimated return lossenhancement is in the range of 0-9 dBm.
 108. The system of claim 91,wherein the at least one processor causes the adaptive filter to have anadaptation step size of 1/32 when a combination of the estimated returnloss and the estimated return loss enhancement is greater than 33 dB.